提出对话事实性评估框架，精准识别大模型幻觉
VISTA: Verification In Sequential Turn-based Assessment
- 将每轮回复拆解为独立事实陈述，逐条验证真伪
- 在8个模型4个数据集上显著优于现有方法
- 适合需要高可信度对话系统的研发与评测
幻觉——指生成与已有证据或对话上下文不符的陈述——仍是部署对话型AI系统时面临的主要障碍。现有评估指标要么仅评价单句回复，要么将无法验证的内容视为错误，难以适用于多轮对话。本文提出VISTA（Verification In Sequential Turn-based Assessment）框架，通过逐条事实验证和对话一致性追踪，评估对话事实性。VISTA将每轮助手回复分解为原子级事实陈述，对照可信来源与对话历史进行验证，并分类未可验证内容（主观、矛盾、无证据或回避）。在8个大型语言模型和4个对话事实性基准（AIS、BEGIN、FAITHDIAL、FADE）上的实验表明，VISTA在幻觉检测上显著优于FACTSCORE和LLM-as-Judge基线。人工评估证实，该分解机制提升了标注者一致性，并暴露了现有基准的不一致问题。通过将事实性建模为对话的动态属性，VISTA提供了更透明、更符合人类认知的对话真实性度量方式。
原文摘要 · Abstract (English)
Hallucination--defined here as generating statements unsupported or contradicted by available evidence or conversational context--remains a major obstacle to deploying conversational AI systems in settings that demand factual reliability. Existing metrics either evaluate isolated responses or treat unverifiable content as errors, limiting their use for multi-turn dialogue. We introduce VISTA (Verification In Sequential Turn-based Assessment), a framework for evaluating conversational factuality through claim-level verification and sequential consistency tracking. VISTA decomposes each assistant turn into atomic factual claims, verifies them against trusted sources and dialogue history, and categorizes unverifiable statements (subjective, contradicted, lacking evidence, or abstaining). Across eight large language models and four dialogue factuality benchmarks (AIS, BEGIN, FAITHDIAL, and FADE), VISTA substantially improves hallucination detection over FACTSCORE and LLM-as-Judge baselines. Human evaluation confirms that VISTA's decomposition improves annotator agreement and reveals inconsistencies in existing benchmarks. By modeling factuality as a dynamic property of conversation, VISTA offers a more transparent, human-aligned measure of truthfulness in dialogue systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。