Anthropic Fable 5.2 completes 41.3% of freelance tasks

Anthropic made a new model, Fable 5.2. It is first on the Remote Labor Index, a test from Scale Labs and the Center for AI Safety. The test measures how many freelance tasks an AI agent completes with a quality that a client will accept. Fable 5.2 completed 41.3 percent. The next systems completed about 21 percent.

How the test works

The test has 240 projects from Upwork, with a value of about $144,000. Agents get a virtual workspace with software such as Blender. Human experts open the files and compare them with the work of the human freelancer.

The numbers

• Fable 5.2: 41.3 percent.
• Opus 5.5: 21.25 percent. GPT Astra 6: 20.83 percent.
• Manus, the top system one year before: 2.5 percent.

What is not known

Scale Labs tells that agents complete almost none of the work in industrial product design and animations. An agent that does not complete the work gives work of low quality, parts that are not full, or broken files. A check by AI does not work, because it finds work that is flawed good.

Insight: An external test gives the proof that Anthropic did not

On November 9 Anthropic made Fable 5.2 20% lower in price. It gave no test scores for coding, reasoning or vision. One day after, an external test gave the number. Anthropic also prepares a public stock sale. This reads as a launch where a third party gives the test result, and a test on your work is the next step.