IEEE checks catch a fake paper that free chatbot tools wrote

Benjamin Skuse, a writer, tried the checks of IEEE Access, a science journal, with the agreement of the journal. Skuse used free ChatGPT and Gemini to write a fake paper on optical solitons. Solitons are light pulses that keep their shape. The work used about 2 dozen prompts. The automated integrity checks of IEEE found fabricated and unverifiable references.

How it works

Skuse did not edit the output. Skuse made the paper in parts: abstract, introduction and methods. When one tool refused, Skuse used the other tool and told it that the next step was an example.

The numbers

The text gives numbers for the full problem:

• In 2025, an estimated 57 percent of academic papers showed signs of LLM influence.
• In 2023, the number was 12 percent.
• For each 40 papers with signs of AI use, only 1 paper disclosed it.

Skuse found no tortured phrases.

What is not known

The text does not give the result of Test 2. In it, Skuse used FARS, an autonomous system that does much of the science. It is not known if IEEE can find a paper from a system that did the research.

Insight: Fake papers fail on references, not on style

The test shows that AI style tics are hard to find, but fabricated references are not. Other tests show that AI tools which examine papers are weak against manipulation, and that research agents agree with pseudoscience. The checks hold against fake papers from free tools, but FARS, which does the research, can change that. Leaders must check which of their checks a machine cannot go through.