根据任务复杂度动态压缩大模型，降低使用成本。
AgentCompress: Task-Aware Compression for Affordable Large Language Model Agents
- 按任务难度智能选择不同压缩程度的模型
- 计算成本降68.3%，成功率仍保持96.2%
- 适合预算有限但需高效推理的团队
大语言模型在多个领域前景广阔，但其计算开销对许多机构构成障碍。单次使用700亿参数模型的云费用约127美元，使资源有限的组织难以负担。我们提出AgentCompress框架，通过任务感知的动态压缩解决此问题。核心思想是：不同任务所需计算量差异显著，复杂推理远比文本重排耗时。传统压缩方法对所有任务一视同仁，而本方案采用轻量级神经控制器，分析请求前几项标记，预估任务复杂度，并将请求路由至相应量化版本的模型。该路由步骤仅增加约12毫秒延迟。我们在涵盖计算机科学、物理、化学和生物领域的290个多阶段工作流上测试，结果表明计算成本降低68.3%，同时保持96.2%的原始成功率。这说明智能路由可显著降低大模型使用门槛而不牺牲输出质量。
原文摘要 · Abstract (English)
Large language models hold considerable promise for various applications, but their computational requirements create a barrier that many institutions cannot overcome. A single session using a 70-billion-parameter model can cost around $127 in cloud computing fees, which puts these tools out of reach for organizations operating on limited budgets. We present AgentCompress, a framework that tackles this problem through task-aware dynamic compression. The idea comes from a simple observation: not all tasks require the same computational effort. Complex reasoning, for example, is far more demanding than text reformatting, yet conventional compression applies the same reduction to both. Our approach uses a lightweight neural controller that looks at the first few tokens of each request, estimates how complex the task will be, and sends it to an appropriately quantized version of the model. This routing step adds only about 12 milliseconds of overhead. We tested the framework on 290 multi-stage workflows from domains including computer science, physics, chemistry, and biology. The results show a 68.3% reduction in computational costs while preserving 96.2% of the original success rate. These findings suggest that routing queries intelligently can make powerful language models substantially more affordable without sacrificing output quality
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。