# Hallucination Detection

This product is an open-source project developed by Vectara to evaluate the hallucination rate of Large Language Models (LLMs) when summarizing short documents. It utilizes Vectara's Hughes Hallucination Evaluation Model (HHEM-2.1) to calculate rankings by detecting hallucinations in the model's output. This tool is significant for researching and developing more reliable LLMs, helping developers understand and improve the accuracy of their models.

The PatronusAI/Llama-3-Patronus-Lynx-70B-Instruct is a large language model built on the Llama-3 architecture, designed to address hallucination issues in RAG settings. By analyzing provided documents, questions, and answers, this model assesses whether the answers are faithful to the document's content. Its primary advantages include high precision in hallucination detection and strong language comprehension capabilities. Developed by Patronus AI, this model is well-suited for scenarios necessitating high-precision information verification, such as financial analysis and medical research. It is currently free to use, but specific commercial applications may require direct contact with the developers.

Llama-3-Patronus-Lynx-8B-Instruct is a fine-tuned version of the meta-llama/Meta-Llama-3-8B-Instruct model developed by Patronus AI, primarily designed to detect hallucinations in retrieval-augmented generation (RAG) settings. The model is trained on multiple datasets, including CovidQA, PubmedQA, DROP, and RAGTruth, featuring both human-annotated and synthetic data. It evaluates whether a given document, question, and answer are faithful to the document content, refraining from providing new information outside the document or contradicting it.