优化工具响应首块内容排序，发现排名首位对代码代理性能无显著影响。
Agents Don't Paginate: First-Chunk Selection for LLM Tool Responses
- 将首块选择建模为0/1背包问题，测试六种价值函数的排序效果。
- 首块包含目标项的精确率仅提升10.8个百分点，但下游准确率未见改善。
- 添加文件元信息反而降低首块命中率，关键词匹配法效果最好但无实际收益。
基于大语言模型的编程代理（如Claude Code、GitHub Copilot等）在处理工具返回结果时常超出单轮令牌预算。尽管协议支持分页，但在公开的Model Context Protocol中间件会话日志中，我们观察到无一例代理主动请求第二块。因此我们关注黄金项（代理所需内容）是否位于首块中的精度-1率（p₁）。在500个SWE-bench Verified任务上，通过离线基准测试将首块选择视为0/1背包问题，比较六种价值函数；再通过单轮文件定位探测（5种语言模型，4,800次调用）检验p₁的影响。两个预注册假设均不成立：第一，提高p₁并未系统性提升下游准确率——各模型差异均低于3个百分点，方向不一致，无显著性；代理能从任意位置恢复目标，关键在于是否进入首块而非其内部排名。第二，加入四个文件元信息信号使p₁下降4.8个百分点（配对t检验，p=0.001）。而一个无需参数的关键词评分器可将p₁从24.2%提升至35.0%（+10.8个百分点，远超随机，p=3.9×10⁻⁸），结合工具原生排序的回退策略可达35.8%，但因核心发现，此提升未转化为下游性能改进。
原文摘要 · Abstract (English)
Coding agents built on large language models (LLMs), such as Claude Code, Cursor, OpenAI Codex, GitHub Copilot, and Aider, receive tool responses that routinely exceed the agent's per-turn token budget. The standard remedy, pagination, is available in every protocol that produced these responses; yet across the corpus of session logs from a public Model Context Protocol middleware we observed no agent-initiated requests for a second chunk. The first chunk is what the agent reads, so we ask how often the gold item (the one the agent needs) is placed first in it: the precision-at-1 rate $p_1$. In a controlled offline benchmark we treat first-chunk selection as a 0/1 knapsack and compare six value functions on 500 SWE-bench Verified tasks, then test whether $p_1$ matters with a single-turn file-localisation probe on five language models (4,800 LLM calls; not an end-to-end resolve-rate test). Two pre-registered hypotheses did not hold and are our main findings. The central one is negative: raising $p_1$ does not systematically raise downstream accuracy. Per-model deltas stay under three percentage points (p.p.), are not consistently signed, and no model is significant; the agent recovers the gold from anywhere in the chunk, so what reaches its answer is first-chunk inclusion, not the gold's rank within it. The second: adding four file-metadata signals to a keyword scorer hurts $p_1$ by 4.8 p.p. (paired significance test, $p = 0.001$). A parameter-free keyword scorer does raise $p_1$, from a 24.2% baseline to 35.0% (+10.8 p.p., far beyond chance; $p = 3.9 \times 10^{-8}$), and to 35.8% with a fallback to the tool's native ordering when no keyword matches. But by our central finding this is a rank-1 gain, and rank-1 is the part that does not reach the agent's answer: downstream accuracy does not move.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。