Research
The ReCast method finds the step that starts an agent failure
ReCast is 5.65 and 9.19 percentage points above the top baseline on 2 Who&When tests.
arxiv.orgClaimed, not confirmed
Tag
arXiv
47 stories carry this tag.
Research
Rules that do not apply in a context file lower the compliance of a model
The authors show that a file with all instructions that help can give a lower total value than a subset.
arxiv.orgClaimed, not confirmed
Research
A new test checks if AI tools find logical errors in papers
The authors put errors in conference papers and show that the systems that check papers are weak against adversarial manipulation.
arxiv.orgClaimed, not confirmed
Research
A new refinement step helps 6 methods of low-bit quantization
The method changes only the integer codes of the weights and adds no inference overhead.
arxiv.orgClaimed, not confirmed
Research
A memory filter cuts the failure rate of an agent by 8.8 percentage points
The paper reports that the 2 filters cause more personalization failures.
arxiv.orgClaimed, not confirmed
Compute
RaReCache lets a small model prefill the cache for a large model
The authors write that the large model keeps 95 to 99% of its precision when it recomputes only 30% of positions.
arxiv.orgClaimed, not confirmed
Security
6 lines of log text give a guardrail a fail-open rate of 63%
Because a deterministic rule was 100% correct on all 6 policies, the model is not necessary.
arxiv.orgClaimed, not confirmed
Benchmarks
An evolved verifier had 0.21 more held-out agreement on MBPP+
Without its guards, the verifier always passes, but the skills that it trains are as good.
arxiv.orgClaimed, not confirmed
Research
GPT-6-astra withdraws the claims of authors when it rewrites papers
The researchers say that less than 1 in 10 of the claims that GPT-6-astra withdraws are overstated.
arxiv.orgClaimed, not confirmed
Benchmarks
HarmBench does not measure one safety attribute, researchers find
3 of the 4 datasets that can target harmful refusal are saturated and cannot show differences between models.
arxiv.orgClaimed, not confirmed
Research
Researchers make TypedBench, a test for typed decision models
The hosted model follows its policy, but it is sensitive to wording and underconfident.
arxiv.orgClaimed, not confirmed
Research
A local model is equal to hosted models for 4 of 5 call sites
The authors tell that 2 hosted call sites give the same live result for 28% of the spend.
arxiv.orgClaimed, not confirmed