Best AI agent gets only 26.7% of hidden rule puzzles right
Agents do worst when they must find a rule by themselves: they got 0.6% right on rule discovery puzzles.
On 18 November 2026, Einsia AI told the public about Visual Cipher Bench. It is a test of AI agents on 40 picture puzzles with hidden rules. The agents must find the rules by themselves. Eight models made 3,480 tries in all, and the test had 29 setups. No setup got more than 30%, and the best, GPT-6 Astra, got 26.7%.
How it works
Each agent gets one instruction: look at the images and write the last answer to a file. The agent must find the examples, the task and the rules by itself. The puzzles include four types: spatial operations, logical planning, rule induction and knowledge decoding.
The numbers
- Claude Opus 5.5 and GPT-6.1 Sol got 21.7%.
- DeepSeek V4.1 Flash, Qwen 3.8 Max and Kimi K3.1 Pro got no puzzle right.
- Spatial puzzles: 22.8%. Rule induction: 0.6%.
With a one-sentence hint on 10 puzzles, accuracy went up from 12.2% to 51.1%.
What is not known
The authors tell that agents stop with rules that fit only some examples. They also try too many paths and time out. The source gives no external inspection of the test, which has only 40 puzzles.
Insight: Top agents are near a quarter on three different tests
GPT-6 Astra leads here with 26.7%. In an earlier coding test it was also best, but it got only 26.4% of tasks fully right. In a different test, the best model got 38.9% of infrastructure tasks right. The pattern reads as a rule: agents are good when a person tells the goal. Who writes the specification gets control. Leaders must keep persons on that task.
Sources
Posted