通过稀疏计算加速推测解码的验证阶段，提升长文本与MoE模型推理效率。
Accelerate Speculative Decoding with Sparse Computation in Verification
- 在验证阶段联合稀疏化注意力、前馈网络和MoE模块
- 实验显示效率提升显著，且保持稳定生成长度
- 无需额外训练，适合长上下文与MoE类大模型场景
推测解码通过并行验证多个草稿令牌加速自回归语言模型推理。然而，验证阶段常成为主要计算瓶颈，尤其在长上下文输入和混合专家（MoE）模型中更为突出。现有稀疏化方法主要针对标准逐令牌自回归解码，以消除大模型中的大量计算冗余。本文系统性地将不同稀疏方法应用于推测解码的验证阶段，识别出跨多个维度的结构化冗余。基于此，提出一种稀疏验证框架，联合稀疏化注意力、前馈网络（FFN）和MoE组件，有效降低主导计算开销。该框架还引入跨草稿令牌与跨层检索结果复用策略，进一步减少冗余计算，且无需额外训练。在摘要、问答和数学推理等多个数据集上的大量实验表明，所提方法在保持稳定接受长度的同时，实现了优良的效率-精度权衡。
原文摘要 · Abstract (English)
Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel. However, the verification stage often becomes the dominant computational bottleneck, especially for long-context inputs and mixture-of-experts (MoE) models. Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding to remove substantial computational redundancy in LLMs. This work systematically adopts different sparse methods on the verification stage of the speculative decoding and identifies structured redundancy across multiple dimensions. Based on these observations, we propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost. The framework further incorporates an inter-draft token and inter-layer retrieval reuse strategy to further reduce redundant computation without introducing additional training. Extensive experiments across summarization, question answering, and mathematical reasoning datasets demonstrate that the proposed methods achieve favorable efficiency-accuracy trade-offs, while maintaining stable acceptance length.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。