arXiv:2603.20432cs.CLcs.AI2026-03被引 10

用编程代理处理长文本,效果比传统方法好17.3%。

Coding Agents are Effective Long-Context Processors

  • 让代码代理通过文件系统和命令行操作长文本
  • 在三万亿标记数据上平均提升17.3%性能
  • 适合需要处理海量文本的开发者或研究者

大语言模型虽能访问海量上下文,但依赖不可解释的注意力机制,导致长文本处理能力随上下文增长显著下降。本文探索将长文本处理从隐式注意力中剥离,转为显式的可执行交互:让现成的前沿代码代理通过文件系统组织文本,并使用原生工具进行操作。评估显示,该方法在长上下文推理、检索增强生成及基于超大规模语料的开放域问答任务中表现优异,平均性能超越已有最先进方法17.3%。其优势源于两项关键因素:对工具的原生熟练度(可执行代码与终端命令),以及对文件系统的熟悉度(将海量文本视为目录结构导航)。这些发现表明,将长文本处理外包给代码代理,是优于语义搜索或扩展上下文窗口的新路径,为大模型长文本处理开辟了新方向。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated remarkable progress in scaling to access massive contexts. However, the access is via the latent and uninterpretable attention mechanisms, and LLMs fail to effective process long context, exhibiting significant performance degradation as context length increases. In this work, we study whether long-context processing can be externalized from latent attention into explicit, executable interactions, by allowing coding agents to organize text in file systems and manipulate it using its native tools. We evaluate off-the-shelf frontier coding agents as the general interface for tasks that require processing long contexts, including long-context reasoning, retrieval-augmented generation, and open-domain question answering with large-scale corpus contains up to three trillion tokens. Across multiple benchmarks, these agents outperform published state-of-the-art by 17.3% on average. We attribute this efficacy to two key factors: native tool proficiency, which enables agents to leverage executable code and terminal commands rather than passive semantic queries, and file system familiarity, which allows them to navigate massive text corpora as directory structures. These findings suggest that delegating long-context processing to coding agents offers an effective alternative to semantic search or context window scaling, opening new directions for long-context processing in LLMs.

代码代理长文本处理文件系统大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。