A test shows plausible AI web forecasts do not give safe decisions
In a check of 3,000 predictions, 86.1 percent looked plausible, but some models chose BLOCK too much and missed cases that needed a question.
Researchers released SafeWebWorld, a test of whether web world models give sufficient information for a safe decision. A web world model predicts what happens after an AI agent takes a step. In a human check of 3,000 predictions from ten open models, 86.1 percent looked plausible. But some models chose BLOCK too much and rarely chose ASK, which means a request for clarification.
How it works
The test gives each step one of three decisions: EXECUTE, ASK or BLOCK. It covers five web safety domains. It checks the predicted future, the evidence that decides the policy, and the last decision.
The proof
Some models showed low false safe execution, but weak recall of ASK cases. Security fine tuning kept more evidence, but made authorization decisions worse. Recalibration of the scores gave a large recovery that is different for each base model.
What is not known
Tests in one model family and on matched cases stay weak. The authors say that plausible prediction, kept evidence and recovery are each not sufficient for robust authorization. The paper gives no exact scores.
Insight: A low error score can hide a model that only blocks
Low false safe execution looks like a good safety score. Here it came with a strong preference for BLOCK and weak recall of ASK cases. A paper of 8 October shows that harmful prompts in safe looking data can attack world models. This seems a trap for buyers, because a score that counts only incorrect executions rewards a model that blocks almost all steps. Ask for ASK recall too.
Sources
Posted
