让大模型自动判断何时重述事实,使推理更准确且更简洁。
REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning

- 根据上下文需求动态重述关键事实,提升推理可靠性。
- 在多个评测集上提升答案准确性,同时减少60%以上的推理token。
- 适合需要高精度、低冗余推理的应用场景。
大型语言模型在解决复杂任务时依赖长流程推理,但当证据不完整、噪声多或与模型参数知识冲突时,推理易偏离上下文。现有方法要么生成后添加引用,要么在推理中检索证据,却难以确保引用信息足以支持中间推断和最终答案。为此,我们提出REFACT——一种自适应事实重述引用框架,使模型能判断何时需要上下文支撑,并选择性地以适当粒度重述源事实,实现可靠推理。通过教师模型构建多样化条件下的高质量引用感知推理轨迹,再用两阶段SFT-to-RL方法优化学生模型。在LongBench、LV-Eval和ConFiQA上的实验表明,REFACT显著提升长上下文问答性能与反事实忠实性,同时大幅减少推理令牌数。分析显示,其通过保留更多相关事实、减少重复陈述,实现更高证据密度,推理过程更紧凑且更可信。所有代码与数据将开源于https://github.com/NEUIR/REFACT。
原文摘要 · Abstract (English)
Large Language Models (LLMs) increasingly leverage long-form reasoning to solve complex tasks, yet their reasoning processes can deviate from the provided context when evidence is incomplete, noisy, or conflicts with parametric knowledge. Existing grounding approaches either append citations after generation or encourage LLMs to retrieve evidence during reasoning, but they often fail to ensure that cited information is sufficient to support intermediate inferences and final answers. To address this limitation, we propose REFACT, an adaptive fact-restatement citation framework that enables LLMs to determine when contextual grounding is needed and selectively restate source facts at appropriate levels of detail for reliable reasoning. To facilitate adaptive citation during reasoning, REFACT first leverages a teacher LLM to construct high-quality citation-aware reasoning trajectories under diverse context conditions with varying evidence lengths, and then optimizes the student LLM through a two-stage SFT-to-RL framework. Experiments on LongBench, LV-Eval, and ConFiQA demonstrate that REFACT improves long-context question answering and counterfactual faithfulness while substantially reducing the number of reasoning tokens. Further analysis reveals that REFACT achieves higher evidence density by preserving more answer-relevant facts with fewer restatements, producing reasoning traces that are more concise yet better grounded. All code and data will be released via https://github.com/NEUIR/REFACT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。