VERITAS提升大模型幻觉检测能力，兼顾效果与效率。
VERITAS: A Unified Approach to Reliability Evaluation
- 构建统一框架，在多种场景下灵活检测幻觉
- 平均性能比同类模型高10%，接近GPT-4 Turbo水平
- 适合需要低成本高可靠性的实际应用部署
大语言模型在知识密集型任务中常因无法从上下文中有效整合信息而生成不准确回答，导致可靠性不足。可靠的LLM需具备鲁棒的事实核查能力，以识别各类格式中的幻觉。尽管已有多个开源核查模型，但功能多局限于特定任务（如基于问答或蕴含验证），在对话场景中表现不佳。而闭源模型如GPT-4和Claude虽更具灵活性，却受限于高昂成本与延迟。本文提出VERITAS，一组可在多种上下文中灵活运行的幻觉检测模型，兼顾低延迟与低成本。在主流幻觉检测基准上，其平均性能较相似规模模型提升10%，在LLM作为裁判者设置下接近GPT-4 Turbo表现。
原文摘要 · Abstract (English)
Large language models (LLMs) often fail to synthesize information from their context to generate an accurate response. This renders them unreliable in knowledge intensive settings where reliability of the output is key. A critical component for reliable LLMs is the integration of a robust fact-checking system that can detect hallucinations across various formats. While several open-access fact-checking models are available, their functionality is often limited to specific tasks, such as grounded question-answering or entailment verification, and they perform less effectively in conversational settings. On the other hand, closed-access models like GPT-4 and Claude offer greater flexibility across different contexts, including grounded dialogue verification, but are hindered by high costs and latency. In this work, we introduce VERITAS, a family of hallucination detection models designed to operate flexibly across diverse contexts while minimizing latency and costs. VERITAS achieves state-of-the-art results considering average performance on all major hallucination detection benchmarks, with $10\%$ increase in average performance when compared to similar-sized models and get close to the performance of GPT4 turbo with LLM-as-a-judge setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。