Researchers jailbreak language models with a moral reframing attack
The success rate is high on the first try and increases with more tries.
Claimed, not confirmed
Researchers made a multi-agent framework that makes a request look good. When a model does not do a request, the agents find the moral value that the request goes against. They find a value that makes the request look good, then rewrite the prompt. The researchers did tests on 5 models. The models were easy to attack with this framework.
Sources
Posted
