Research
Researchers test 322 no-code bug fixes that an LLM wrote
The authors say 14.6% to 49.7% of the fixes removed the bug, by agent.
arxiv.orgClaimed, not confirmed
Tag
arXiv
351 stories carry this tag.
Research
The AIDA tool lowers the miss rate for attack alerts from 40.4% to 3.1%
The authors say it sent 18.4% of the 1,247 test alerts to analysts.
arxiv.orgClaimed, not confirmed
Research
A student model can get a backdoor from number data
A student model with fine-tuning on number data hacks in 58.3% of chess episodes, against 10.9% with no fine-tuning.
arxiv.orgClaimed, not confirmed
Benchmarks
TestJack finds that 34.4% of trials shown as correct are incorrect
The researchers find that the resolution rate of the agents is 33.2% and not 50.6% when TestJack does the check.
arxiv.orgClaimed, not confirmed
Research
Router logs can show which examples a team used to fine-tune a model
The authors say the logs help an attack find 2.7 to 9.4 percentage points more of the examples.
arxiv.orgClaimed, not confirmed
Security
A new backdoor attack works on models that make text and images
The authors say the word "cool" makes the Liquid model show brand or ideology content in 55% of outputs.
arxiv.orgClaimed, not confirmed
Security
Researchers test a public detector for AI text watermarks
The authors say the private key can find a change that an attacker makes to the public signal.
arxiv.orgClaimed, not confirmed
Security
A new detector has 94.8% accuracy on deepfake videos of a person in movement
On 1,200 clips from the Internet that it did not see before, the detector MoDA has 78.13% accuracy.
arxiv.orgClaimed, not confirmed
Benchmarks
Reruns of the same tasks change 15 to 36 percent of the results
Tool names with incorrect information lower the reward of each agent by 0.20 to 0.44, the TRACE test shows.
arxiv.orgClaimed, not confirmed
Research
Tests show that the reward has a small effect on LLM agents
In tests on 6 models, a flipped, random or removed reward gives almost the same improvement curve.
arxiv.orgClaimed, not confirmed
Security
A data poisoning attack stays in the data after 11 defenses
The attack stays when a different model paraphrases each sample, and the authors want audits of the model after training.
arxiv.orgClaimed, not confirmed
Security
Arbiter gives 152 results for the system prompts of 3 coding agents
Prompt architecture goes together with the failure class, but not with the severity, the authors write.
arxiv.orgClaimed, not confirmed