arXiv:2605.01106cs.CLcs.AI2026-05

让混合模型内部组件自发预判,提速推理并揭示架构关键差异。

Component-Aware Self-Speculative Decoding in Hybrid Language Models

  • 利用混合模型中不同组件的结构差异,以内部子图零成本生成候选词。
  • 并行混合模型接受率高达68%,而串行模型仅3.8%,差距达18倍。
  • 架构融合方式决定能否自洽预测,适合研究高效推理与模型设计者。

推测解码通过快速模型生成候选词并并行验证来加速自回归推理。自推测方法无需外部草稿模型,但以往仅在同质Transformer架构中研究。本文提出组件感知的自推测解码,首次利用混合语言模型内部架构异构性,将SSM/线性注意力子图作为零成本内部草稿。在两种架构迥异的混合模型Falcon-H1(每层并行:Mamba-2 + attention)和Qwen3.5(串行:交替线性与注意力层)上评估,对比纯Transformer基线(Qwen2.5)。并行混合模型在贪婪解码下,草稿长度k=2时接受率α=0.68;串行混合模型仅α=0.038,差距达18倍,归因于组件集成方式。该特性具有尺度不变性:Falcon-H1在3B规模复现0.5B的接受率。进一步发现,伴随消融实验中的困惑度退化比可预测推测可行性:Falcon的3.15倍比对应α=0.37(k=4),Qwen的81.96倍比对应α=0.019。对串行混合模型,通用的LayerSkip策略接受率是组件感知策略的12倍。混合模型的组合模式——而非仅存在替代组件——决定组件级自推测是否可行。

原文摘要 · Abstract (English)

Speculative decoding accelerates autoregressive inference by drafting candidate tokens with a fast model and verifying them in parallel with the target. Self-speculative methods avoid the need for an external drafter but have been studied exclusively in homogeneous Transformer architectures. We introduce component-aware self-speculative decoding, the first method to exploit the internal architectural heterogeneity of hybrid language models, isolating the SSM/linear-attention subgraph as a zero-cost internal draft. We evaluate this on two architecturally distinct hybrid families: Falcon-H1 (parallel: Mamba-2 + attention per layer) and Qwen3.5 (sequential: interleaved linear and attention layers), with a pure Transformer control (Qwen2.5). Parallel hybrids achieve acceptance rates of alpha = 0.68 at draft length k=2 under greedy decoding, while sequential hybrids yield only alpha = 0.038 -- an 18x gap attributable to how each architecture integrates its components. The property is scale-invariant: Falcon-H1 at 3B reproduces the rates observed at 0.5B. We further show that perplexity degradation from a companion ablation study predicts speculative viability without running speculative decoding: a 3.15x ratio (Falcon) maps to alpha = 0.37 at k=4, while 81.96x (Qwen) maps to alpha = 0.019. For sequential hybrids, generic LayerSkip achieves 12x higher acceptance rates than the component-aware strategy. The composition pattern of hybrid models -- not merely the presence of alternative components -- determines whether component-level self-speculation is viable.

推理加速混合架构自推测模型设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。