让大模型在推理时主动用好上下文信息，提升回答一致性。
Improving Context Fidelity via Native Retrieval-Augmented Reasoning
- 用模型自身检索能力显式整合上下文证据进行推理
- 在多个基准上显著优于微调和外部检索方法
- 只需少量标注数据，适合实际知识密集型任务
大语言模型常因上下文保真度不足，导致基于给定信息作答时出现不一致。现有方法或依赖昂贵的监督微调生成答案后证据，或训练模型进行网络搜索，但未必提升对已有上下文的利用。我们提出CARE——一种原生检索增强推理框架，通过模型自身检索能力，在推理链中显式集成上下文证据。该方法仅需少量标注证据数据，即可通过策略性检索上下文标记，显著提升检索准确率与生成性能。在多个真实世界及反事实问答基准上的实验表明，该方法大幅超越监督微调、传统检索增强生成及外部检索方案。本工作为提升大模型在知识密集型任务中的准确性、可靠性与效率提供了根本性进展。
原文摘要 · Abstract (English)
Large language models (LLMs) often struggle with context fidelity, producing inconsistent answers when responding to questions based on provided information. Existing approaches either rely on expensive supervised fine-tuning to generate evidence post-answer or train models to perform web searches without necessarily improving utilization of the given context. We propose CARE, a novel native retrieval-augmented reasoning framework that teaches LLMs to explicitly integrate in-context evidence within their reasoning process with the model's own retrieval capabilities. Our method requires limited labeled evidence data while significantly enhancing both retrieval accuracy and answer generation performance through strategically retrieved in-context tokens in the reasoning chain. Extensive experiments on multiple real-world and counterfactual QA benchmarks demonstrate that our approach substantially outperforms supervised fine-tuning, traditional retrieval-augmented generation methods, and external retrieval solutions. This work represents a fundamental advancement in making LLMs more accurate, reliable, and efficient for knowledge-intensive tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。