What matters in AI.

Subscribe

Research

Vision language models are 20.0 points less accurate on physical cues

On average, the 5 models are 42.7% accurate with a physical cue and 62.7% accurate with scene context, and humans are 63.4% and 68.5% accurate.

Claimed, not confirmed

VLMs show a 20.0-point gap between direct and inferential images; humans do not Bars on a 0 to 100 scale. Five VLMs: 42.7% on direct images (shadow, reflection) and 62.7% on inferential images (scene context), a 20.0-point gap. Humans: 63.4% direct and 68.5% inferential, no significant gap. Five VLMs Humans Direct shadow, reflection 42.7% 63.4% Inferential scene context 62.7% 68.5% 20.0-point gap No significant gap
Five vision-language models do far worse when a shadow or reflection gives away a hidden object. People do about equally well on both kinds.

Sources

  1. OccludeBench: Can VLMs Find What's Missing?arxiv.org
AI MATTER · NEWS · AI MATTER · NEWS ·12 OCT2026

Posted

Tags

Learn the terms in this story

More in Research

All Research news