Tests show 4 of 4 AI providers open to attack in code pipelines

Researchers released GitInject, an open-source framework that tests prompt injection in live GitHub Actions workflows. Prompt injection is hidden text that makes an AI agent follow the attacker. The team tested 11 attack scenarios across 5 workflows from 4 AI providers. Each provider had one or more high-severity attacks in the tested setups, where external contributions start agents with no human approval.

How it works

GitInject makes temporary repositories and operates live workflows, not simulated tool calls. This shows sandbox limits, credential handling and permission limits. The attacks are of 4 types:

• configuration-file injection
• credential theft
• judgment manipulation
• availability attacks

The numbers

The authors find that the worst holes are structural. Configuration files from an unmerged contribution install as operator-level instructions. 2 of these attacks also work on GitLab and Hugging Face, and not only on GitHub. The authors tested workflow-level countermeasures and show the tradeoffs in coverage and use.

What is not known

The summary gives no success rate for each provider and does not name them. The tests include only fully automated setups. It does not tell the cost of each countermeasure.

Insight: Model training does not close this hole

An earlier test found that Meta SecAlign, which has training against prompt injection, had a 32% attack success rate. GitInject finds its worst holes in the pipeline, where files from an unmerged contribution install as operator instructions. This reads as a move of the risk from the model to the pipeline. The team that controls which inputs an agent trusts holds the control. Monitor pipeline rules, not only model changes.