Most AI models keep one known solution when they select goals
Most of the models kept one known solution, but humans selected many different goals.
Claimed, not confirmed
Researchers did a test of 5 AI models on a task from cognitive science. In the task, a learner selects the goals. Humans found new goals step by step, and each human was different. Most models kept one known solution, or gave low results. The researchers write that these models must not replace humans in goal selection.
How it works
The task has a learner that selects the goals. The 5 models were:
The numbers
The study gives no scores. Each model showed a different pattern. Copies of the same model were almost the same. A chain of thought (the model writes its steps) and persona steering (the model plays a role) caused only a small change.
What is not known
Tests in applied work are not done at this time. The authors write that the results stay the same in all their test conditions.
Insight: Agents are weakest at the selection of goals
A study found that most of 5 models kept one known solution when they selected goals. Taste-Bench tests long tasks. The top of 14 models got 59.7% on the selection of the right step. This reads as a weak point in the work that firms give to agents first: goal selection. Leaders keep control if a human stays in that selection.
Sources
Posted
