A simulator gives AI models 97% and 100% accuracy on Apollo code

The authors of a new paper made tests on two Claude AI models, Sonnet and Haiku, with the Apollo 11 flight code. A simulator is a tool that operates the code. With it, Sonnet gave correct answers to 100% of code questions and Haiku to 97%. Without it, they gave 81% and 48%. The authors found a problem. A model can give the answer for a known program, and not read the code in front of it.

How it works

The authors made a test from nine Apollo programs. They removed names and comments. In other tests, they changed one instruction, and the known answer became incorrect. The models had three setups: file search, a code graph, and a simulator.

The numbers

Without the simulator, the models gave the initial answer in 6% to 17% of tests after the change. With it, Sonnet did that in 0 of 64 tests and Haiku in 1 of 64. The code graph did not change the measured result. Python rewrites were fully correct on all 32,768 inputs in at most 4 of 9 tests. More gave the correct result in sample tests.

What is not known

The test used two models from one company, one agent setup and 27 routines. The simulator shows that models can check an answer. It does not show that they know the code.

Insight: Sample tests are weak proof of agent work

A different open source tool, REA, lets agents examine programs. In one example, it gave the same result as all 3205 test cases of a game function. In the Apollo test, six rewrites gave the correct result in all sample tests. All were incorrect on the same 77 inputs. This reads as a sign that the party with the full check sets the trust in agent output.