AI Hallucination

ELI5 (Explain like I’m 5)

Imagine a friend who’s great at telling stories. If you ask, “What happened in that movie?”, they might guess parts they don’t remember, but they’ll say it with confidence. And at times, they may even come up with things that didn’t happen in the movie.

AI models can do something similar. They don’t look up facts by default. They are designed to generate the next most likely words. If the model doesn’t truly know something, it may still produce an answer that sounds correct, even when it isn’t. That’s referred to as a hallucination.

Why do AI Models Hallucinate?

Hallucinations arise from core LLM limitations.

1. Prediction instead of verification

Most LLMs, especially the text-based chat models, are trained to predict the next likely word, not to verify truth like a search engine. That makes them good at fluent explanations, and vulnerable to making up details when the prompt pushes for certainty.

As per OpenAI’s research paper, the way the models are trained, giving an answer even if incorrect is rewarded over not giving an answer. So the model comes up with something of its own, if it doesn’t know the answer.

2. Missing context or Training Data Limitations

When a model lacks enough context, or is asked about niche, recent, or ambiguous topics, it may respond with best guesses. Similarly, if the training data is incorrect, biased, or contains inconsistencies, the model may respond with the wrong answer.

3. Lack of Real-World Understanding

Unlike humans, AI models don't possess common sense or an understanding of the physical world. They operate based on statistical relationships learned from text. This makes them generate outputs that can be grammatically correct but semantically nonsensical in a real-world context.

E.g.:

Confidence increased as certainty became unavailable. Context was preserved by discarding all relevant information.

How to Reduce AI Hallucinations?

While completely eliminating AI hallucinations remains a challenge, the below strategies can help reduce their occurrence.

1. Retrieval-Augmented Generation (RAG)

This involves connecting LLMs to external, verified knowledge bases. Before generating a response, the model retrieves relevant information from these reliable sources and grounds its answer in those facts, significantly reducing the likelihood of making up things.

2. Improved Training Data and Techniques

Using higher quality, less biased and more diverse training data can help to an extent. Fact-checking techniques should be included in the training process. The training should also include confidence scoring to estimate the model’s confidence in its response and use it as a way to discourage making up answers if the model doesn’t know something.

3. Human Review and Feedback

Implementing human review for critical applications helps in identification and correction of hallucinations.

As AI becomes more integrated into critical applications, addressing hallucinations is crucial for building trust and ensuring reliable performance.

There are also people who want these models to come up with things which otherwise they wouldn’t have done, there you are exactly right. Hallucination is a feature, not a bug.

— Sundar Pichai