Benchmarks
Reruns of the same tasks change 15 to 36 percent of the results
Tool names with incorrect information lower the reward of each agent by 0.20 to 0.44, the TRACE test shows.
arxiv.orgClaimed, not confirmed
News category
Benchmarks
17 stories, newest first.
Benchmarks
An evolved verifier had 0.21 more held-out agreement on MBPP+
Without its guards, the verifier always passes, but the skills that it trains are as good.
arxiv.orgClaimed, not confirmed
Benchmarks
HarmBench does not measure one safety attribute, researchers find
3 of the 4 datasets that can target harmful refusal are saturated and cannot show differences between models.
arxiv.orgClaimed, not confirmed
Benchmarks
Grok 4.7 is first in the new legal AI test with 9.4%
Artificial Analysis says a hallucination is material if it can mislead a reader, for example with an incorrect contract date.
Benchmarks
AI agents wrote GPU code with 1.57× the speed of `torch.compile`
The author tells that the correct test was more work than the fast code.
towardsdatascience.comClaimed, not confirmed