让大模型递归处理超长文本，突破上下文限制。
Recursive Language Models
- 将长文本视为外部环境，递归调用自身分段处理
- 处理长度超模型窗口两个数量级，性能提升最高130%
- 适合需要长文本理解的复杂任务，如代码生成与推理
我们从推理阶段扩展的角度研究如何让大语言模型（LLMs）处理任意长的提示。提出递归语言模型（RLMs），一种通用的推理范式，将长提示视为外部环境，允许模型程序化地检查、分解并递归调用自身处理提示片段。实验发现，RLMs可成功处理超出模型上下文窗口两个数量级的输入；即使在较短提示下，其性能也显著优于原始前沿模型及常见的长上下文和编码框架（例如，在GPT-5上，相比压缩方法中位提升26%，相比CodeAct带子调用提升130%，相比Claude Code提升13%），且成本相当。我们在小规模上对首个基于RLM的模型进行后训练，该模型RLM-Qwen3-8B在四个多样化长上下文任务上平均比原Qwen3-8B提升28.3%，并在三个任务上接近原生GPT-5表现。代码已开源于https://github.com/alexzhang13/rlm。
原文摘要 · Abstract (English)
We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive Language Models (RLMs), a general inference paradigm that treats long prompts as part of an external environment and allows the LLM to programmatically examine, decompose, and recursively call itself over snippets of the prompt. We find that RLMs can successfully process inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of vanilla frontier LLMs and common long-context and coding scaffolds (e.g., on GPT-5 by a median across the evaluated benchmarks of $26\%$ against compaction, $130\%$ against CodeAct with sub-calls, and $13\%$ against Claude Code) across four diverse long-context tasks while having comparable cost. At a small scale, we post-train the first model around the RLM. Our model, RLM-Qwen3-8B, outperforms the underlying Qwen3-8B model by $28.3\%$ on average and even approaches the quality of vanilla GPT-5 on three long-context tasks. Code is available at https://github.com/alexzhang13/rlm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。