Defined term
Hallucination
Plausible but factually incorrect output generated by an LLM with no grounding.
Hallucination is when a model produces an answer that sounds confident but is not supported by the provided sources or the real world. Hallucination cannot be eliminated by prompt alone; production defenses combine grounding (RAG), confidence scoring, citation requirements, output validation, and human review for low-confidence cases.
When it matters
Highest-stakes failure mode in any AI workflow. A hallucinated number, citation, or fact can ship a wrong contract, a wrong dose, a wrong claim decision. Defense-in-depth required: grounding + validation + reviewer queues.
Real example
A model citing 'NIST SP 800-53 control AC-22' when the document was about AC-21 — a single-digit hallucination that would have passed a tired reviewer. Caught by a citation-validator that re-fetched the source and flagged the mismatch.
KPIs to watch
Hallucination rate on factual claims (<1% target for production), citation-validator catch rate (>95%), reviewer override rate trending up (signal that hallucinations are slipping through).
Related terms
Grounding
Anchoring model output to verifiable source material to reduce hallucination.
RAG (Retrieval-Augmented Generation)
Generation grounded in retrieved source documents rather than the model's parametric memory alone.
Guardrails
Pre and post checks that filter unsafe, off-topic, or non-compliant model outputs.
Prompt injection
An attack where user input manipulates the model into ignoring its system prompt or executing unintended actions.
See it in action
We use this every week
Book a 30-min call and we'll walk you through how Hallucination shows up in a real engagement we're running.
Book a 30-min call