研究长文本中信息分布如何影响大模型的推理与幻觉。
Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs
- 设计可扩展的寻针测试基准，分离评估提取、推理与幻觉风险。
- 发现事实分散时模型性能显著下降，抗幻觉提示导致过度拒绝真实信息。
- 适合关注长文本推理可靠性的研究人员和部署者参考。
随着大语言模型越来越多地利用超长上下文窗口作为自主任务的工作内存，其可靠性在很大程度上取决于现实语料中信息的分布方式。本文通过引入一个模型无关的可扩展寻针基准，研究了事实位置、语料级分布以及反幻觉提示（'不要编造'）对模型行为的影响，并应用于Gemini-2.5-flash、ChatGPT-5-mini、Claude-4.5-haiku和Deepseek-v3.2-chat的评估。不同于以往工作，我们分别评估了字面提取、逻辑推理和幻觉风险。识别出两种关键失败模式：分布坍塌（Distributional Collapse），即当证据分散时性能大幅下降；以及安全税（Safety Tax），即反幻觉提示导致模型过度保守地拒绝存在的真实事实与证据，显著降低准确率。结果表明，许多失败源于无效的上下文利用，模型即使面对已有相关信息也难以有效优先处理。这些发现强调了针对具体模型提升鲁棒性及优化上下文管理的必要性，以确保在长周期智能体工作流中的可靠部署。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) increasingly utilize massive context windows as working memory for autonomous tasks, their reliability fluctuates significantly depending on how information is distributed in real-world corpora. We investigate how fact placement, corpus-level distributions, and anti-hallucination ("Don't Make It Up") prompts influence model behavior by introducing a model-agnostic extended needle-in-a-haystack benchmark designed for scalability, which we apply to evaluate Gemini-2.5-flash, ChatGPT-5-mini, Claude-4.5-haiku, and Deepseek-v3.2-chat. Unlike prior work, we separately evaluate literal extraction, logical inference, and hallucination risk. We identify two critical failure modes: Distributional Collapse, where performance degrades significantly when evidence is dispersed; and a Safety Tax, where anti-hallucination prompts cause over-conservative refusal of present facts and evidence, sharply reducing accuracy. Our results suggest that many failures stem from ineffective context utilization, as models struggle to prioritize relevant information even when it is present. These findings highlight the need for model-specific robustness and effective context management to ensure reliable deployment in long-horizon agentic workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。