All 11 AI humanizer tools changed facts in 35% to 65% of documents
Tools that rewrite AI text to avoid detectors change quoted text, numbers and names, and the leaderboard does not score this.
Researchers checked 11 commercial AI humanizers (tools that rewrite AI-written text to make detectors tell it is human) in the October 2026 cycle of a open leaderboard. Each tool changed or removed a number, date, name or quoted text in 35% to 65% of the documents that had one. For numbers and names only, 10 of the 11 tools changed one in 20% or more of the documents.
How it was tested
A rule-based checker looked for numbers, dates, names and quoted text in each rewrite. 2 AI annotators validated the checker. The team used no human annotation.
The numbers
An open 12B humanizer kept 97% or more of the numbers and names in US federal agency text. It reworded 67% to 78% of the quoted text. A decision model, Jev, separated human texts from humanized texts with a score of 0.864, but it said that 24% of human texts are AI.
What is not known
Retention of facts and evasion of detectors were not clearly connected. 2 perplexity-based detectors had a score below chance on AI-styled rewrites by 2026 models. The authors release the checker and the results of each item.
Insight: No one scores if the text is correct
The leaderboard scores evasion and similarity, not facts. A different paper found that systems that check papers are weak against adversarial manipulation. It looks like a contest where humanizers get a score for evasion and detectors get a score for finding AI text, and no one scores if the text is correct. Look for a fact check as a standard score.
Sources
Posted
