测试大模型在错误上下文中的忠实度，发现越大模型也不一定更靠谱。
FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"
- 构建三类挑战性上下文测试模型忠实性
- 4900个高质量问题验证模型表现
- 揭示大模型仍易幻觉，适合评测可信AI
确保大语言模型（LLMs）和检索增强生成（RAG）系统在真实应用中对上下文的忠实性至关重要，因不准确或无依据的信息会削弱用户信任。尽管基准测试有进展，模型在上下文不符时仍常产生幻觉。本文提出FaithEval，一个针对三种典型情境（不可回答、不一致、反事实）的综合性评估基准，模拟检索系统可能返回不完整、矛盾或虚构信息的真实挑战。该基准包含总计4900个经四阶段严格构建与验证的高质量问题，结合大模型自动评估与人工校验。对众多开源及专有模型的广泛研究显示，即使顶尖模型也常无法保持上下文忠实，且模型规模增大并不必然提升忠实度。
原文摘要 · Abstract (English)
Ensuring faithfulness to context in large language models (LLMs) and retrieval-augmented generation (RAG) systems is crucial for reliable deployment in real-world applications, as incorrect or unsupported information can erode user trust. Despite advancements on standard benchmarks, faithfulness hallucination-where models generate responses misaligned with the provided context-remains a significant challenge. In this work, we introduce FaithEval, a novel and comprehensive benchmark tailored to evaluate the faithfulness of LLMs in contextual scenarios across three diverse tasks: unanswerable, inconsistent, and counterfactual contexts. These tasks simulate real-world challenges where retrieval mechanisms may surface incomplete, contradictory, or fabricated information. FaithEval comprises 4.9K high-quality problems in total, validated through a rigorous four-stage context construction and validation framework, employing both LLM-based auto-evaluation and human validation. Our extensive study across a wide range of open-source and proprietary models reveals that even state-of-the-art models often struggle to remain faithful to the given context, and that larger models do not necessarily exhibit improved faithfulness.Project is available at: https://github.com/SalesforceAIResearch/FaithEval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。