测试大模型在协调内容占用上下文时的任务表现下降现象
RCWT: Measuring Task-Budget Displacement from Coordination Content in LLM Calls

- 设计受控实验测量协调内容对任务预算的挤占效应
- 当剩余证据降至几百词时,模型性能骤降
- 证明问题本质是预算分配失衡,非协调量本身干扰
多智能体与记忆增强型大模型系统常将协调内容、共享状态、过往讨论、工具输出、摘要和角色指令等置于同一有限上下文窗口中。这导致实际分配难题:每个用于协调的词元都会减少可用于任务指令或证据的词元。本文提出轮桌上下文窗口测试(RCWT),一种受控协议来测量此任务-预算挤占效应。在 $W=4096$ 的主要上下文依赖回忆任务中,三种商用模型在中等协调开销下保持基线水平,一旦剩余参考证据降至数百词,性能急剧下降。窗口缩放摘要支持任务特定残余预算解释,而非固定百分比阈值,但仅作为描述性证据。为检验当任务证据完整时是否仍存在固定预算悬崖,引入完整任务消融实验:保留完整任务/参考块,通过扩展提示长度增加协调词元。在此设定下,GPT-4.1-mini、Claude Haiku 4.5 和 Gemini 2.5 Flash 在高达 95% 协调比例时均正确返回所有评分字段。该消融缩小了结论:主RCWT悬崖应解读为任务预算挤占,而非协调量本身引起语义干扰。因此,RCWT是上下文分配预算的测量基础,而非多智能体优势或会话级协调的完整理论。
原文摘要 · Abstract (English)
Multi-agent and memory-augmented LLM systems often place coordination content, shared state, prior discussion, tool outputs, summaries, and role instructions, inside the same finite prompt used for the current task. This creates a practical allocation problem: every token spent on coordination is unavailable to task instructions or evidence when a call is assembled under a fixed context budget. We introduce the Roundtable Context Window Test (RCWT), a controlled protocol for measuring this task-budget displacement effect. RCWT varies coordination content while controlling total budget, position order, task family, and scoring. In the main context-dependent recall task at $W=4096$, three commercial models remain near baseline through moderate overhead and then degrade sharply once residual reference evidence falls to a few hundred tokens. Window-scaling summaries are consistent with a task-specific residual-budget interpretation rather than a fixed percentage threshold, but we treat this as descriptive evidence rather than a universal law. To test whether the fixed-budget cliff persists when task evidence remains intact, we add an intact-task ablation: the full task/reference block is kept present while coordination tokens increase by expanding total prompt length. In that setting, all tested calls return every scored field correctly across GPT-4.1-mini, Claude Haiku 4.5, and Gemini 2.5 Flash up to a 95\% coordination ratio. This ablation narrows the claim: the main RCWT cliff is best read as task-budget displacement, not as proof that coordination volume alone causes semantic interference in the original open-ended task. RCWT is therefore a measurement primitive for context-allocation budgeting, not a complete theory of multi-agent benefit or session-level coordination.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。