让大模型推理更快更准，关键在前后处理用不同上下文。
AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs
- 轻量草稿模型读全上下文，大模型验证时用压缩版，打破传统对称性。
- 平均达全上下文90%准确率，吞吐提升1.3至1.7倍，算力降低一半。
- 适合需要多轮交互和工具调用的智能体系统，尤其在上下文压缩时保精度。
智能体大模型在检索、工具调用和多轮交互中累积上下文，导致推理成本飙升。为控制延迟，部署常压缩输入，但会降低任务准确性。推测解码（SD）能无损加速生成，但其假设草稿与验证模型共享相同上下文，无法解决准确率与开销的权衡。本文提出AsymSpec，一种非对称推测解码框架：轻量级草稿模型读取完整输入，而大型验证模型基于压缩视图运行。草稿通过对比性δ-融合对数概率引导验证器，由感知差异的接受门控调节，确保验证稳定性与高草稿接受率。在四个智能体能力及两个端到端代理基准上评估，AsymSpec平均达到全上下文90%的准确率，在孤立文本任务上实现1.3–1.7倍吞吐提升，计算成本降至0.2–0.3倍。结果表明，当压缩丢弃关键推理信号时，非对称上下文访问可带来显著收益。
原文摘要 · Abstract (English)
Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the accuracy-overhead trade-off. We propose AsymSpec, an asymmetric speculative decoding framework that breaks this symmetry: a lightweight drafter reads the full input while the large verifier operates on the compressed view. The drafter steers the verifier via a contrastive $δ$-fusion of logits, modulated by a divergence-aware acceptance gate that preserves verification stability and high draft acceptance rates. Evaluated across four agentic capabilities and two end-to-end agent benchmarks, AsymSpec reaches $\approx 90\%$ of full-context accuracy on average, delivering $1.3$--$1.7\times$ throughput speedups at $0.2$--$0.3\times$ the compute cost on isolated text capabilities. These results show that asymmetric context access yields substantial gains precisely when compression discards critical reasoning signals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。