A8gent
GlossaryAgent concepts

What is hallucination?

Also called: confabulation, AI making things up

A hallucination is output that is fluent, confident and wrong - a language model producing plausible detail that has no basis in its input or in fact.

Written by A8gent
Contents
  1. At a glance
  2. What a hallucination is
  3. Why it happens
  4. What actually reduces it
  5. What you should know
  6. FAQ
  7. Related terms

At a glance

Category
Core agent concept
Two kinds
Closed-book (invented from training) and open-book (contradicts supplied sources)
Root cause
Models predict plausible continuations; nothing marks a token as true
Biggest single fix
Ground the answer in retrieved source text and require verifiable quotes
Second fix
Give the model an explicit permitted way to say it does not know
Not a fix
Temperature 0, or telling the model in the prompt not to hallucinate

What a hallucination is

A hallucination is model output that reads as authoritative and is not true: an invented order number, a policy clause that does not exist, a citation to a paper nobody wrote. The word is contested - some researchers prefer confabulation - but the operational meaning is stable and it is the single biggest reason businesses hesitate to put a model in front of customers.

It helps to split the phenomenon in two. Closed-book hallucination is the model inventing something it was never given, drawing on statistical patterns from training. Open-book hallucination is the model contradicting or embellishing source material that was right there in the prompt. The second is the one that shows up in business automations, and it is far more fixable.

Why it happens

A language model predicts likely continuations. Nothing in that process distinguishes "true" from "plausible", and a specific-sounding answer is often more probable than an admission of ignorance, because the training data is full of confident answers and thin on people saying they do not know. Under pressure - a question whose answer is not in the context, a schema demanding a field the source never mentioned - producing something is the path of least resistance.

This is why prompt design matters more than model choice for this particular failure. A prompt that says "answer from the supplied documents; if the answer is not there, reply exactly NOT_FOUND" gives the model a permitted way out. A prompt that just asks the question does not, and a schema with a required field is actively demanding invention.

What actually reduces it

Four things, in order of impact for a typical build. First, ground the answer: retrieval-augmented generation puts the real source text in the prompt so the model is summarising rather than recalling. Second, require citations - make the model quote the sentence it relied on, and validate that the quote appears in the source. That single check catches most open-book hallucinations mechanically.

Third, allow abstention and make it cheap: an explicit NOT_FOUND path, optional rather than required schema fields, and a route to a human when the model abstains. Fourth, verify anything with a canonical source: an order number should be checked against the orders table by a tool call, not trusted because it looks right. Lower temperature helps a little at the margins but is not a fix.

What you should know

Hallucination cannot currently be eliminated, only made rare and detectable. That shapes what you should automate: tasks where a wrong answer is caught by a downstream check are safe, tasks where a wrong answer goes straight to a customer are not, unless a person approves. Measure the rate rather than debating it - a twenty-case eval with fact-check assertions will tell you your real number in an afternoon.

Be careful with the word in front of clients too. "The AI hallucinated" sounds like an excuse; "the system produced an unverified value and our validation step caught it before it reached you" describes a working control, which is what a well-built automation should be able to say.

Hallucination FAQ

Why do AI models hallucinate?

Because they generate the most likely next token, and likelihood is not truth. Training data is full of confident answers and short on people saying they do not know, so a specific-sounding invention often scores higher than an admission of ignorance - especially when your prompt or schema demands an answer.

Does RAG stop hallucinations?

It reduces them substantially by putting the real text in front of the model, so the task becomes summarising rather than recalling. It does not eliminate them: models still embellish supplied sources, and if retrieval returns nothing relevant the model may answer from memory anyway. Pair retrieval with required citations.

Does setting temperature to zero prevent hallucinations?

No. Temperature 0 makes the output deterministic, not correct - it will reliably reproduce the same confident invention. It is worth setting for extraction and classification because it removes variance, but it should never be presented as an accuracy control.

How do I detect hallucinations automatically?

Make the model quote the exact source sentence it relied on, then check that string appears in the source document. Verify any identifier - order numbers, SKUs, customer IDs - against the real system with a lookup. Both are cheap deterministic checks that catch the majority of open-book cases.

What tasks are too risky for an AI that might hallucinate?

Anything where a wrong answer reaches a customer or a regulator with no check in between: medical, legal or financial advice, warranty and refund promises, delivery commitments. Those need a human approval step. Tasks where a downstream validation catches errors are safe to automate fully.

Do bigger models hallucinate less?

Generally yes on general knowledge, but the difference on your specific task is often smaller than the difference good grounding and validation make. A small model reading the right document usually beats a frontier model guessing, at a fraction of the cost. Test both on your own eval set.

Primary source

A Survey on Hallucination in Large Language Models (arXiv)

Checked August 2026. Anything priced or rate-limited moves, so verify against the source before you budget.

Want this built rather than explained?

Knowing what hallucination means is one thing; having it running in your business is another. We scope one workflow at a time, quote a fixed price in writing after a free consult, build it, and hand it over with documentation you own.