压缩提示词时,输出变长会拉高成本,需综合权衡。
Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial
- 测试了不同压缩率和结构感知策略对成本影响
- 中等压缩(保留50%)降低总成本27.9%,激进压缩反而增本1.8%
- 输出长度变化是关键,适合生产部署优化提示策略
提示词压缩的经济性不仅取决于输入词数减少,更受输出长度变化影响,后者通常定价更高。我们在真实多智能体任务编排场景中开展预注册的六臂随机对照试验,分析358次成功的Claude Sonnet 4.5运行(每组59-61次),来自1,199条真实编排指令的随机语料。比较未压缩对照组与三种均匀保留率(r=0.8, 0.5, 0.2)及两种结构感知策略(熵自适应、时效加权),测量总推理成本(输入+输出)和基于嵌入的响应相似度。中等压缩(r=0.5)使均值总成本降低27.9%;激进压缩(r=0.2)虽显著减少输入,但均值成本反升1.8%,尽管输出仅轻微扩展1.03倍,且存在重尾不确定性。时效加权压缩实现23.5%成本节约,与中等压缩共同占据经验成本-相似度帕累托前沿;激进压缩则在成本与相似度上均被超越。结果表明,“压缩越多越好”并非可靠生产策略,设计压缩政策时必须将输出词数视为首要考量。
原文摘要 · Abstract (English)
The economics of prompt compression depend not only on reducing input tokens but on how compression changes output length, which is typically priced several times higher. We evaluate this in a pre-registered six-arm randomized controlled trial of prompt compression on production multi-agent task-orchestration, analyzing 358 successful Claude Sonnet 4.5 runs (59-61 per arm) drawn from a randomized corpus of 1,199 real orchestration instructions. We compare an uncompressed control with three uniform retention rates (r=0.8, 0.5, 0.2) and two structure-aware strategies (entropy-adaptive and recency-weighted), measuring total inference cost (input+output) and embedding-based response similarity. Moderate compression (r=0.5) reduced mean total cost by 27.9%, while aggressive compression (r=0.2) increased mean cost by 1.8% despite substantial input reduction, consistent with small mean output expansion (1.03x vs. control) and heavy-tailed uncertainty. Recency-weighted compression achieved 23.5% savings and, together with moderate compression, occupied the empirical cost-similarity Pareto frontier, whereas aggressive compression was dominated on both cost and similarity. These results show that "compress more" is not a reliable production heuristic and that output tokens must be treated as a first-class outcome when designing compression policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。