发现大模型会遗漏或曲解复杂指令，导致意图幻觉。
Beyond Facts: Evaluating Intent Hallucination in Large Language Models
- 提出新概念'意图幻觉'，指模型忽略或误解查询条件。
- 构建20,068题的FAITHQA基准，覆盖多种任务场景。
- 设计自动评估指标CONSTRAINT SCORE，更贴近人类判断。
当面对包含多个条件的复杂查询时，当前大型语言模型（LLMs）往往只部分满足查询要求，忽略某些条件。为此，我们引入了‘意图幻觉’这一概念：即模型在生成时遗漏（未处理查询中的某些部分）或误读（回应虚构的查询内容），导致生成结果偏离原始意图。为系统评估该现象，我们提出了首个超越事实验证的基准——FAITHQA，包含20,068个问题，涵盖仅查询和检索增强生成（RAG）两种设置，涉及不同主题与难度。通过在FAITHQA上评估多种大模型，我们发现：（1）即使最先进的模型也普遍存在意图幻觉；（2）该现象源于模型对查询条件的遗漏或误判。为促进后续研究，我们提出一种自动评估指标CONSTRAINT SCORE，用于检测意图幻觉。人工评估显示，该指标在识别意图幻觉方面比基线方法更接近人类表现。
原文摘要 · Abstract (English)
When exposed to complex queries containing multiple conditions, today's large language models (LLMs) tend to produce responses that only partially satisfy the query while neglecting certain conditions. We therefore introduce the concept of Intent Hallucination. In this phenomenon, LLMs either omit (neglecting to address certain parts) or misinterpret (responding to invented query parts) elements of the given query, leading to intent hallucinated generation. To systematically evaluate intent hallucination, we introduce FAITHQA, a novel benchmark for intent hallucination that contains 20,068 problems, covering both query-only and retrieval-augmented generation (RAG) setups with varying topics and difficulty. FAITHQA is the first hallucination benchmark that goes beyond factual verification, tailored to identify the fundamental cause of intent hallucination. By evaluating various LLMs on FAITHQA, we find that (1) intent hallucination is a common issue even for state-of-the-art models, and (2) the phenomenon stems from omission or misinterpretation of LLMs. To facilitate future research, we introduce an automatic LLM generation evaluation metric, CONSTRAINT SCORE, for detecting intent hallucination. Human evaluation results demonstrate that CONSTRAINT SCORE is closer to human performance for intent hallucination compared to baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。