What matters in AI.

Subscribe

Best AI agent gets only 26.7% of hidden rule puzzles right

Agents do worst when they must find a rule by themselves: they got 0.6% right on rule discovery puzzles.

On 18 November 2026, Einsia AI told the public about Visual Cipher Bench. It is a test of AI agents on 40 picture puzzles with hidden rules. The agents must find the rules by themselves. Eight models made 3,480 tries in all, and the test had 29 setups. No setup got more than 30%, and the best, GPT-6 Astra, got 26.7%.

How it works

Each agent gets one instruction: look at the images and write the last answer to a file. The agent must find the examples, the task and the rules by itself. The puzzles include four types: spatial operations, logical planning, rule induction and knowledge decoding.

The numbers

  • Claude Opus 5.5 and GPT-6.1 Sol got 21.7%.
  • DeepSeek V4.1 Flash, Qwen 3.8 Max and Kimi K3.1 Pro got no puzzle right.
  • Spatial puzzles: 22.8%. Rule induction: 0.6%.

With a one-sentence hint on 10 puzzles, accuracy went up from 12.2% to 51.1%.

What is not known

The authors tell that agents stop with rules that fit only some examples. They also try too many paths and time out. The source gives no external inspection of the test, which has only 40 puzzles.

Insight: Top agents are near a quarter on three different tests

GPT-6 Astra leads here with 26.7%. In an earlier coding test it was also best, but it got only 26.4% of tasks fully right. In a different test, the best model got 38.9% of infrastructure tasks right. The pattern reads as a rule: agents are good when a person tells the goal. Who writes the specification gets control. Leaders must keep persons on that task.

Sources

  1. Einsia AI Launches Visual Cipher Bench to Evaluate Long-Horizon Visual Reasoning in AI AgentsThe Manila Times
AI MATTER · NEWS · AI MATTER · NEWS ·18 NOV2026

Posted

Tags

Learn the terms in this story

More in Research

All Research news