Large Language Models don’t "hallucinate" in the human sense; they’re statistical machines that generate sequences of words based on probability, and sometimes those probabilities lead them down paths that don’t align with factual reality.
Let’s see this in action. Imagine we have a simple RAG (Retrieval Augmented Generation) system. We want to ask a model about a specific fictional event, "The Great Biscuit Shortage of 2023," and we have a document that describes it.
Example Document (context.txt):
The Great Biscuit Shortage of 2023 was a pivotal moment in culinary history. It began on March 15th when a sudden, unexplained fungal blight devastated wheat crops across the United Kingdom. Bakeries worldwide reported unprecedented stockouts within days. The crisis led to widespread panic-buying of cookies, crackers, and other baked goods. Governments enacted emergency measures, including rationing and the promotion of alternative snacks like rice cakes and fruit. The shortage lasted for 18 months, finally easing in September 2024 as new blight-resistant wheat strains were developed.
Now, let’s craft a prompt.
Basic Prompt:
You are a helpful assistant. Answer the following question based on the provided context.
Context:
The Great Biscuit Shortage of 2023 was a pivotal moment in culinary history. It began on March 15th when a sudden, unexplained fungal blight devastated wheat crops across the United Kingdom. Bakeries worldwide reported unprecedented stockouts within days. The crisis led to widespread panic-buying of cookies, crackers, and other baked goods. Governments enacted emergency measures, including rationing and the promotion of alternative snacks like rice cakes and fruit. The shortage lasted for 18 months, finally easing in September 2024 as new blight-resistant wheat strains were developed.
Question: When did the Great Biscuit Shortage of 2023 end?
Expected Output (from a well-behaved LLM):
The Great Biscuit Shortage of 2023 ended in September 2024.
This works because the answer is directly present in the text. But what if we ask a slightly different question, or if the context had a subtle contradiction?
Prompt with Ambiguity:
You are a helpful assistant. Answer the following question based on the provided context.
Context:
The Great Biscuit Shortage of 2023 was a pivotal moment in culinary history. It began on March 15th when a sudden, unexplained fungal blight devastated wheat crops across the United Kingdom. Bakeries worldwide reported unprecedented stockouts within days. The crisis led to widespread panic-buying of cookies, crackers, and other baked goods. Governments enacted emergency measures, including rationing and the promotion of alternative snacks like rice cakes and fruit. The shortage lasted for 18 months, finally easing in September 2024 as new blight-resistant wheat strains were developed.
Question: What was the primary cause of the Great Biscuit Shortage of 2023?
Potential "Hallucinated" Output (if the model over-generalizes or fabricates):
The primary cause of the Great Biscuit Shortage of 2023 was a global economic downturn that impacted supply chains.
Here, the model invented a cause not present in the text. The prompt engineering anti-hallucination techniques aim to prevent these divergences.
The core problem is that LLMs are designed to predict the next most likely word. Without strong guidance, this can lead them to generate plausible-sounding but incorrect information, especially when the prompt is underspecified, the context is ambiguous, or the model’s training data contains conflicting information. Anti-hallucination techniques steer the model towards factual grounding and away from speculative generation.
The primary mechanism is Constraining the Output Space. Instead of letting the LLM freely generate text, we provide it with strong signals about what kind of answer is expected and where to find it. This involves structuring the prompt, carefully selecting the context, and sometimes even telling the model how to reason.
The fundamental levers you control are:
- Context Quality and Relevance: Providing accurate, concise, and directly relevant information.
- Prompt Structure and Instruction: Giving clear, unambiguous instructions on how to use the context.
- Model Parameters: Adjusting settings like
temperatureto control randomness. - Output Validation: Implementing checks on the generated response.
Let’s dive into specific techniques.
1. Few-Shot Learning with Examples: This is about showing the model what a good answer looks like.
Prompt:
You are a helpful assistant that answers questions based *only* on the provided context. If the answer is not in the context, state "Information not available in context."
Context:
The Great Biscuit Shortage of 2023 was a pivotal moment in culinary history. It began on March 15th when a sudden, unexplained fungal blight devastated wheat crops across the United Kingdom. Bakeries worldwide reported unprecedented stockouts within days. The crisis led to widespread panic-buying of cookies, crackers, and other baked goods. Governments enacted emergency measures, including rationing and the promotion of alternative snacks like rice cakes and fruit. The shortage lasted for 18 months, finally easing in September 2024 as new blight-resistant wheat strains were developed.
Question: When did the Great Biscuit Shortage of 2023 end?
Answer: The Great Biscuit Shortage of 2023 ended in September 2024.
Question: What was the primary cause of the Great Biscuit Shortage of 2023?
Answer: The primary cause was a sudden, unexplained fungal blight that devastated wheat crops.
Question: Which countries were most affected by the shortage?
Answer: Bakeries worldwide reported stockouts, and governments enacted emergency measures, suggesting a global impact.
Question: What was the capital of France during the shortage?
Answer: Information not available in context.
By providing examples of both direct answers and the "information not available" response, you train the model on the desired behavior within the prompt itself. This is incredibly powerful for implicitly defining the boundaries of acceptable answers.
2. Explicit Negative Constraints: Tell the model what not to do.
Prompt:
You are a helpful assistant. Answer the following question based *only* on the provided context. Do not infer, speculate, or use external knowledge. If the answer is not directly present in the context, state "The context does not provide this information."
Context:
[Same context as above]
Question: What was the primary cause of the Great Biscuit Shortage of 2023?
The phrases "Do not infer, speculate, or use external knowledge" and "If the answer is not directly present…" are critical. They explicitly forbid the model from venturing outside the provided text, directly addressing the mechanism of hallucination by cutting off the pathways of speculation.
3. Chain-of-Thought (CoT) and Step-by-Step Reasoning: For more complex questions, breaking down the reasoning process can reveal where hallucinations might occur and force grounding.
Prompt:
You are a helpful assistant. Answer the following question by following these steps:
1. Identify the key entities and dates in the question.
2. Scan the provided context for information related to these entities and dates.
3. Synthesize the relevant information from the context to form an answer.
4. If the information is not found in step 2, state "The context does not provide this information."
5. Do not use any external knowledge.
Context:
[Same context as above]
Question: How long did the Great Biscuit Shortage of 2023 last?
The model is now instructed to perform specific, verifiable steps. If step 2 fails to find information about the duration, step 4 is triggered. This structured approach makes it harder for the model to "jump" to a conclusion.
4. Retrieval Augmented Generation (RAG) Specifics: If you’re using RAG, the quality of retrieval is paramount. If the retrieval system fails to fetch the correct document or relevant snippets, the LLM has nothing to ground its answer on.
- Chunking Strategy: How you break down your documents into smaller, retrievable pieces. Too large, and relevant info might be diluted; too small, and context is lost.
- Embedding Model: The quality of vector representations matters.
- Similarity Threshold: Setting an appropriate score (e.g.,
0.75cosine similarity) to decide if a retrieved chunk is relevant enough to include in the context window. If the threshold is too low, irrelevant chunks can confuse the LLM. If too high, you might miss relevant information.
5. Model Parameters (temperature):
The temperature parameter controls the randomness of the output. A temperature of 0.0 makes the output deterministic and more focused on high-probability words.
Example API Call Snippet (Python):
response = openai.Completion.create(
model="text-davinci-003",
prompt=your_prompt,
temperature=0.0, # Set to 0 for maximum determinism
max_tokens=150
)
Setting temperature=0.0 is a blunt but effective tool. It forces the model to pick the most likely next token, significantly reducing the chances of it going off-script. However, it can also make the output sound repetitive or less creative. A common practice is to use a very low temperature (e.g., 0.1 or 0.2) for factual Q&A.
6. Output Post-Processing and Validation: After the LLM generates a response, you can run checks.
- Keyword Spotting: Does the answer contain keywords expected from the context?
- Fact Verification: For critical applications, use a separate LLM or a rule-based system to verify claims against the source.
- Consistency Checks: If the LLM generates multiple related pieces of information, are they consistent with each other and the source?
The most subtle, yet often most effective, anti-hallucination technique is defining the "world" of the LLM with extreme precision through prompt context and instructions, and then ruthlessly punishing any deviation. This means not just providing the facts, but also explicitly stating the rules of engagement: "You MUST use ONLY this information," "You MUST NOT guess," "If it’s not here, say so." When the LLM is in a "zero-shot" or "few-shot" context where its vast pre-training knowledge is intentionally suppressed in favor of the provided text, it’s much less likely to invent. The key is that the model’s internal "decision-making" process, while probabilistic, is heavily biased by these very strong, explicit instructions in the prompt, overriding its general tendencies to predict plausible continuations.
The next hurdle you’ll likely face is handling questions where the context itself contains contradictions or insufficient detail to give a definitive answer, forcing you to refine your strategy for ambiguity.