AI hallucination

How it works

NIST AI 600-1 says confabulations “are a natural result of the way generative models are designed”: they “approximate the statistical distribution of their training data”, which can produce accurate text and can also produce text that is “factually inaccurate or internally inconsistent”.

Kalai and colleagues at OpenAI and Georgia Tech (2025) argue that hallucination starts in pretraining, the first stage where a model learns from a large body of text, and survives because of how models are graded. Some facts, such as birthdays, follow no pattern a model could learn. If 20 percent of those facts appear exactly once in the training data, they expect a freshly pretrained model to hallucinate on at least 20 percent of them. Most benchmarks then give a point for a right answer and nothing for “I don’t know”, so guessing scores better than admitting uncertainty.

Why it matters

A hallucination does harm when someone acts on it. NIST says risks “may arise when users believe false content – often due to the confident nature of the response”. OWASP’s 2026 list says the core risk is that “the incorrect output is trusted and acted upon”.

In code, the invented fact can be a package name. Spracklen et al. tested models on their own Python and JavaScript prompts in 2024. On average, at least 5.2 percent of the packages recommended by the three commercial GPT models, and 21.7 percent of those from open-source models, did not exist. An attacker who registers such a name gets their code run by anyone who trusts the suggestion: slopsquatting. OWASP names hallucination as a trigger for excessive agency, and warns that incorrect output can pass from one AI agent to another, which is how cascading failures start.

Where definitions disagree

Standards bodies and researchers disagree on the name, and on where the problem sits.

NIST calls it confabulation. NIST AI 600-1 (July 2024) treats “hallucinations” and “fabrications” as colloquial names. Its definition also covers output that strays from the prompt or contradicts the model’s earlier statements, which is wider than a false fact.

OWASP has moved it between risks. Version 1.1 of OWASP’s Top 10 for applications built on large language models put it under LLM09 Overreliance, where the risk was people or systems trusting the output unchecked. In 2025, LLM09 became Misinformation. It calls hallucination one of the major causes, says bias and incomplete information can also contribute, and treats overreliance as a related issue. In 2026, Misinformation became LLM07. It still lists hallucinated packages, but says registering those names as an attack belongs under LLM04 Supply Chain.

Some researchers reject the word. Maleki, Padmanabhan and Dutta (2024) reviewed 333 definitions from 14 databases and found that “a formal and consistent definition of hallucination simply does not currently exist”. They report the objection from medical researchers that the metaphor is a misnomer, because a model has no sensory perception, and that it is stigmatising, because it ties AI errors to a symptom of mental illness. They note that “confabulation”, the word NIST uses, has mental-health connections too. They name fabrication, stochastic parroting and hasty generalization as viable alternatives.

Questions and answers

How do I handle AI hallucinations in a business report?

Treat every factual claim, figure and citation an AI tool puts in the report as unchecked until someone has matched it to an authoritative source. OWASP's 2025 guidance is to cross-check model output against trusted external sources, and its 2026 guidance adds keeping verified facts separate from assumptions. NIST warns that models can also invent the citations that appear to back a claim, so check that each cited source exists and says what the report says it does.

Is an AI hallucination the same as confabulation?

Yes, in practice. NIST AI 600-1 uses confabulation as the name of the risk and says the same thing is colloquially called hallucination or fabrication. NIST's definition is slightly wider than a false fact: it also covers output that strays from the prompt or contradicts what the model said earlier in the same conversation.

Why do AI models hallucinate?

NIST says generative models approximate the statistical distribution of their training data, which can produce accurate output and can also produce false output. Researchers at OpenAI and Georgia Tech argue that training and evaluation also reward guessing, because most benchmarks give no credit for answering "I don't know".