There are two broad approaches for detecting hallucinations:
1. Verify the correctness of the response against world knowledge (via Google/Bing search)
2. Verify the groundedness of the response against the information present in the retrieved context
The 2nd approach is more interesting and useful as the majority of LLM applications have an RAG component, and we ideally want the LLM only to utilize the retrieved knowledge to generate the response.
While researching state-of-the-art techniques on how to verify that the response is grounded wrt context, two of the papers stood out to us:
1. FactScore (https://arxiv.org/pdf/2305.14251.pdf): Developed by researchers at UW, UMass Amherst, Allen AI and Meta, it first breaks down the response into a series of independent facts and independently verifies if each of them.
2. Automatic Evaluation of Attribution by LLMs (https://arxiv.org/pdf/2305.06311.pdf): Developed by researchers at Ohio State University, it prompts the LLM judge to determine whether the response is attributable (can be verified), extrapolatory (unclear) or contradictory (can\u2019t be verified).
While both the papers are awesome reads, you can observe that they tackle complementary problems and, hence, can be combined for superior performance:
1. The responses in production systems typically consist of multiple assertions; hence, breaking them into facts, evaluating them individually, and taking average is a more practical approach.
2. Many responses in production systems fall in the grey area, i.e. the context may not explicitly support (or disprove) them but one can make a reasonable argument to infer them from the context. Hence, having three options - Yes, No, Unclear is a more practical approach
This is exactly what we do at UpTrain to evaluate factual accuracy. Learn more about it: https://docs.uptrain.ai/predefined-evaluations/context-awareness/factual-accuracy"},"title":{"fullyHighlighted":false,"matchLevel":"partial","matchedWords":["hallucination"],"value":"How does one detect hallucinations?"}},"_tags":["story","author_sourabh03agr","story_39454961","ask_hn"],"author":"sourabh03agr","children":[39456129],"created_at":"2024-02-21T15:27:36Z","created_at_i":1708529256,"num_comments":2,"objectID":"39454961","points":5,"story_id":39454961,"story_text":"Hallucinations are an interesting artifact of LLMs where the model tends to make up facts or generate outputs that are not factually correct.
There are two broad approaches for detecting hallucinations:
1. Verify the correctness of the response against world knowledge (via Google/Bing search)
2. Verify the groundedness of the response against the information present in the retrieved context
The 2nd approach is more interesting and useful as the majority of LLM applications have an RAG component, and we ideally want the LLM only to utilize the retrieved knowledge to generate the response.
While researching state-of-the-art techniques on how to verify that the response is grounded wrt context, two of the papers stood out to us:
1. FactScore (https://arxiv.org/pdf/2305.14251.pdf): Developed by researchers at UW, UMass Amherst, Allen AI and Meta, it first breaks down the response into a series of independent facts and independently verifies if each of them.
2. Automatic Evaluation of Attribution by LLMs (https://arxiv.org/pdf/2305.06311.pdf): Developed by researchers at Ohio State University, it prompts the LLM judge to determine whether the response is attributable (can be verified), extrapolatory (unclear) or contradictory (can\u2019t be verified).
While both the papers are awesome reads, you can observe that they tackle complementary problems and, hence, can be combined for superior performance:
1. The responses in production systems typically consist of multiple assertions; hence, breaking them into facts, evaluating them individually, and taking average is a more practical approach.
2. Many responses in production systems fall in the grey area, i.e. the context may not explicitly support (or disprove) them but one can make a reasonable argument to infer them from the context. Hence, having three options - Yes, No, Unclear is a more practical approach
This is exactly what we do at UpTrain to evaluate factual accuracy. Learn more about it: https://docs.uptrain.ai/predefined-evaluations/context-awareness/factual-accuracy","title":"How does one detect hallucinations?","updated_at":"2024-09-20T16:27:19Z"},{"_highlightResult":{"author":{"matchLevel":"none","matchedWords":[],"value":"alexostrovskyy"},"comment_text":{"fullyHighlighted":false,"matchLevel":"full","matchedWords":["hallucination","rag","groundedness"],"value":"I think many of us have felt the pain of building a cool LLM-powered application or RAG pipeline, only to find it's too brittle and unpredictable for real-world use. The core problem is that they are black boxes. When they fail, it's hard to know why.
I've been focused on this problem of "productionizing" AI workflows. It's not just about testing; it's about deep observability, performance tuning, and building systems you can trust to be stable.
I wrote up a guide on a methodology I've found very effective. It's based on an open-source framework that uses decorators to trace the entire execution path of a chatbot. This gives you the data to:
- Pinpoint Performance Bottlenecks: See the exact latency of every LLM call, tool use, and retrieval step.\n- Automate Quality Control: Use an LLM-as-a-judge to programmatically check for hallucinations (groundedness), safety violations, and adherence to custom rules.\n- Create a Feedback Loop for Improvement: When you change a prompt or logic, you can run the test suite and get a concrete report on whether performance and reliability have improved or worsened.
You can read the guide here: \n- LangChain-based application: https://alexostrovskyy.com/the-glass-box-why-your-chatbot-ne...,\n- LlamaIndex-based application: https://alexostrovskyy.com/production-llm-chatbot-tracing-an...
I\u2019ve created this open-source project to use in my projects and help other creators.
My goal is to create a framework (open-source) that can help us build stable, trustworthy AI systems, not just clever demos.
I'd be very interested to hear feedback from other engineers and creators."},"story_title":{"matchLevel":"none","matchedWords":[],"value":"An Open-Source Framework for Building Stable and Reliable LLM-Powered Systems"},"story_url":{"matchLevel":"none","matchedWords":[],"value":"https://chatbot-testing-framework.readthedocs.io/en/latest/"}},"_tags":["comment","author_alexostrovskyy","story_45445710"],"author":"alexostrovskyy","comment_text":"I think many of us have felt the pain of building a cool LLM-powered application or RAG pipeline, only to find it's too brittle and unpredictable for real-world use. The core problem is that they are black boxes. When they fail, it's hard to know why.
I've been focused on this problem of "productionizing" AI workflows. It's not just about testing; it's about deep observability, performance tuning, and building systems you can trust to be stable.
I wrote up a guide on a methodology I've found very effective. It's based on an open-source framework that uses decorators to trace the entire execution path of a chatbot. This gives you the data to:
- Pinpoint Performance Bottlenecks: See the exact latency of every LLM call, tool use, and retrieval step.\n- Automate Quality Control: Use an LLM-as-a-judge to programmatically check for hallucinations (groundedness), safety violations, and adherence to custom rules.\n- Create a Feedback Loop for Improvement: When you change a prompt or logic, you can run the test suite and get a concrete report on whether performance and reliability have improved or worsened.
You can read the guide here: \n- LangChain-based application: https://alexostrovskyy.com/the-glass-box-why-your-chatbot-ne...,\n- LlamaIndex-based application: https://alexostrovskyy.com/production-llm-chatbot-tracing-an...
I\u2019ve created this open-source project to use in my projects and help other creators.
My goal is to create a framework (open-source) that can help us build stable, trustworthy AI systems, not just clever demos.
I'd be very interested to hear feedback from other engineers and creators.","created_at":"2025-10-02T02:08:02Z","created_at_i":1759370882,"objectID":"45445711","parent_id":45445710,"story_id":45445710,"story_title":"An Open-Source Framework for Building Stable and Reliable LLM-Powered Systems","story_url":"https://chatbot-testing-framework.readthedocs.io/en/latest/","updated_at":"2026-03-05T22:48:22Z"}],"hitsPerPage":20,"nbHits":2,"nbPages":1,"page":0,"params":"query=hallucination+RAG+groundedness&advancedSyntax=true&analyticsTags=backend","processingTimeMS":13,"processingTimingsMS":{"_request":{"queue":1,"roundTrip":20},"fetch":{"query":11,"total":12},"total":13},"query":"hallucination RAG groundedness","serverTimeMS":14}