让大模型更有效利用长文本中的信息,无需训练即可提升推理能力。
ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning

- 用内部相关性信号构建查询相关的证据池并递归重播。
- 在128K上下文长度下,三种主流模型平均排名最优。
- 适用于需精准长文本推理的场景,如法律、医疗分析。
理解与推理长篇上下文已成为大语言模型在实际应用中的关键需求。尽管近期模型支持越来越长的上下文窗口,但常无法有效利用输入中已存在的相关信息,暴露出上下文访问与实际使用之间的差距。本文提出一种无需训练的推理方法——递归证据重播(RECONTEXT),通过模型内部的相关性信号构建查询条件下的证据池,并在最终生成前重播该池,同时保留完整原始上下文。该递归选择过程将证据组织与答案生成分离,无需训练、外部记忆或上下文裁剪。我们基于联想记忆理论进行分析,将上下文视为记忆存储,问题作为检索线索,注意力视为线索-痕迹关联,重播则为痕迹再激活。在八项长上下文数据集(128K上下文长度)上的实验表明,RECONTEXT在Qwen3-4B、Qwen3-8B和Llama3-8B三种骨干模型上均持续提升证据利用率,三项模型平均排名最佳。代码已开源:https://github.com/Yanjun-Zhao/ReContext。
原文摘要 · Abstract (English)
Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation. Experiments on eight long-context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3-4B, Qwen3-8B, and Llama3-8B, achieving the best average rank on all three backbones. Code is available at https://github.com/Yanjun-Zhao/ReContext.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。