解决大文档生成时模型沉默问题,通过容量分析与分步渲染提升效率
When Agents Go Quiet: Output Generation Capacity and Format-Cost Separation for LLM Document Synthesis
- 提出输出生成容量(OGC)衡量模型真实输出能力,低于上下文窗口
- 证明延迟模板渲染比直接生成节省48%-72%令牌,消除生成中断
- 设计自适应策略选择框架,适合需要高效生成复杂文档的开发者
基于大模型的编码代理在生成大型格式化文档时常出现无声失败现象,称为输出停滞。本文提出理论框架,包含三项贡献:(1) 引入输出生成容量(OGC),量化代理在当前上下文状态下的实际输出能力,该值小于原始上下文窗口;(2) 证明格式成本分离定理,显示对于任意格式开销倍数μ_f > 1,延迟模板渲染在所有条件下至少与直接生成同样高效,并推导出精确的节省边界;(3) 形式化自适应策略选择机制,根据预估输出成本与可用OGC的比率,自动决策采用直接、分块或延迟生成策略。在三种模型(Claude 3.5 Sonnet、GPT-4o、Llama 3.1 70B)、四种文档类型及消融实验中验证理论有效性。延迟渲染使生成令牌减少48%-72%,彻底消除输出停滞。将框架实现为开源MCP服务器GEN-PILOT,证明理论可直接转化为实用工具。
原文摘要 · Abstract (English)
LLM-powered coding agents suffer from a poorly understood failure mode we term output stalling: the agent silently produces empty responses when attempting to generate large, format-heavy documents. We present a theoretical framework that explains and prevents this failure through three contributions. (1) We introduce Output Generation Capacity (OGC), a formal measure of an agent's effective ability to produce output given its current context state - distinct from and empirically smaller than the raw context window. (2) We prove a Format-Cost Separation Theorem showing that deferred template rendering is always at least as token-efficient as direct generation for any format with overhead multiplier $μ_f > 1$, and derive tight bounds on the savings. (3) We formalize Adaptive Strategy Selection, a decision framework that maps the ratio of estimated output cost to available OGC into an optimal generation strategy (direct, chunked, or deferred). We validate the theory through controlled experiments across three models (Claude 3.5 Sonnet, GPT-4o, Llama 3.1 70B), four document types, and an ablation study isolating each component's contribution. Deferred rendering reduces LLM generation tokens by 48-72% across all conditions and eliminates output stalling entirely. We instantiate the framework as GEN-PILOT, an open-source MCP server, demonstrating that the theory translates directly into a practical tool.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。