Participants in a public challenge wrote 208,095 attack prompts
Meta SecAlign has training against prompt injection, but it has a 32% attack success rate.
Claimed, not confirmed
Researchers made LLMail-Inject, a benchmark of 208,095 attack prompts that humans wrote. The prompts come from a public challenge with 839 participants. An attack must go from an email database through production defenses. It must also cause a tool call that is not approved, with the correct arguments. GPT-4o has a 58% attack success rate, and GPT-5 mini has 47%.
Sources
Posted
