通过预执行多步动作链,显著降低工具型大模型代理的延迟。
Speculative Macro Commit for Faster Tool-Using Agents
- 用快速推测模型预执行未来动作序列,匹配后批量提交结果。
- 在电信任务上比串行执行快18.59%,在应用世界任务上快44.9%。
- 适合追求高响应速度的智能代理系统,尤其适用于重复性任务场景。
工具型大模型代理不仅耗时于模型推理,还因串行的动作-观测循环而产生延迟。本文提出一种运行时机制——投机式宏提交(SMC),用于双层代理系统:由大型权威模型生成正式轨迹,快速推测模型则在隔离环境快照中持续预测并执行未来动作链。SMC从训练轨迹中挖掘重复出现的多动作模式,并存入宏库,在运行时用于匹配推测模型预测的动作链。当权威模型下一步动作与推测首步匹配时,便将剩余预执行步骤及其观测一并提交至正式轨迹。使用Qwen3.5-27B INT4作为权威模型、Qwen3.5-4B作为推测模型,在τ²-Bench Telecom子集上,准确率与串行代理相当,但延迟降低10.23%(相比推测动作基线)和18.59%(相比串行执行);在AppWorld上,延迟减少7.7%(相比基线)和44.9%(相比串行),任务完成率略有下降。整体上,SMC提供了一种可复用多步推测执行的实用方法,有效降低代理延迟。
原文摘要 · Abstract (English)
Tool-using LLM agents spend wall-clock time not only on model inference but also in serial action--observation turns, where each tool call, environment transition, and observation can delay subsequent decisions. We introduce \textbf{Speculative Macro Commit} (SMC), a runtime mechanism for a two-tier agent system: a large authoritative actor model produces the official trajectory, while a faster speculative drafter model continuously predicts and executes future action chains on an isolated environment snapshot. SMC mines recurring multi-action skeletons from training traces and stores them in a macro library used to match against action chains predicted by the drafter at runtime. When the actor's next tool call matches the first drafted action, SMC commits the remaining pre-executed draft steps, together with their observations, to the official trajectory. Using Qwen3.5-27B INT4 as the authoritative actor model and Qwen3.5-4B as the speculative drafter model, SMC matches the sequential agent's overall accuracy while reducing latency by 10.23\% over the Speculative Actions (SA) baseline and 18.59\% over sequential execution on the $\tau^2$-Bench Telecom subset. On AppWorld, SMC reduces wall time by 7.7\% over SA baseline and 44.9\% over sequential execution, with a small reduction in task completion. Overall, SMC provides a practical way to reuse multi-step speculative execution and reduce agent latency beyond single-step speculative actions. Our code is publicly available \href{https://github.com/zeyuliu1037/speculative-macro-commit}{\textcolor{magenta}{here}}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。