Study of 14 AI models finds 6 patterns that make text hard to read

Researchers at New York University and the University of Texas at Austin did tests on 14 language models in casual chat, coding sessions and essays. They compared 120 conversations, 45 coding sessions and 980 essays with human text. The models use rare word groups, put related parts of a sentence far apart, and turn nouns into verbs. The researchers tell that training for math and code is the likely cause.

What they found

The models put many nouns in a row, as in “horror movie soundtrack generator”. They write “calendar it” where a human writes “put it in the calendar”. Readers can find the meaning, but they use more time.

The proof

Llama and the Olmo base model are closest to human text. Scores in the Olmo family increase from the base model to the reasoning model. This points to training after the base model as the likely cause.

What is not known

The cause is an argument of the researchers, not a fact. They tell that training with human ratings was replaced by training with automatic checks, which give no reward for clarity. Their test of this used one small model with a reward for short answers. It selected short phrasing almost always, and humans selected by context.

Insight: Labs reward what a machine can check, so clarity was lost

A related study of 9 models also connects hard to read text to training for math and code. An Anthropic employee, Jackson Kernion, tells that newer models moved to a style made for AI readers. The next model changed part of this back. This reads as a trade: machines can check math and code, not clarity, and one lab chose to repair it. Look for which labs copy the repair.