AI models had at most 63.5% on research judgment, researchers 73.4%
4 of 5 frontier models had scores below chance on the selected cases.
The SciPivot benchmark has 248 cases from 6 months of research. In that time, senior researchers operated LLM agents on 26 projects and corrected their research decisions. With the project record, the accuracy of the models in the test was at most 63.5%. New researchers had 73.4%.
Sources
Posted
