通过修复高不确定性节点,提升长文本生成的并行推测解码效率。
CURE: Local Uncertainty Repair for Block-Parallel Speculative Decoding

- 基于置信度边缘动态定位错误候选词,仅在脆弱节点扩展修复路径。
- 在多个基准上使接受长度提升4.2%-7.5%,端到端提速2.66-3.49倍。
- 可插拔设计,适合追求低延迟长文本生成的研究与工程团队。
推测解码通过交错草案生成与目标验证,缓解自回归大模型的序列生成延迟。然而,现有并行草案后端在长序列上常因准确率快速下降导致高拒绝率,影响实际加速效果。我们观察到,草案错误并非均匀分布,而是集中于局部高不确定性标记,这些标记会扰乱下游生成轨迹。为此,提出CURE:一种预算感知的动态修复树机制,可在不引入过高验证开销的前提下,精准修复不确定性焦点处的错误。具体而言,该方法利用预测置信度边缘动态定位块并行草案中的候选错误标记,在这些脆弱节点展开有限修复路径,并采用新型修复重同步机制在验证后对齐草案状态。在代码生成基准(HumanEval、MBPP、LiveCodeBench-lite)和数学推理基准(GSM8K)上的评估表明,相较于无修复的并行基线,CURE将平均接受长度提升4.2%-7.5%,实现相对于目标仅解码的2.66-3.49倍端到端加速。此外,我们提供一个兼容标准并行草案框架的即插即用修复模块,并量化分析了草案计算与验证效率间的权衡。
原文摘要 · Abstract (English)
Speculative decoding mitigates the latency of sequential generation in autoregressive Large Language Models (LLMs) by interleaving draft generation with target verification. However, existing parallel drafting backends often suffer from rapid accuracy degradation over long horizons, leading to high rejection rates during verification and suboptimal wall-clock speedups. We observe that drafting errors are not uniformly distributed but typically stem from localized high-uncertainty tokens that destabilize downstream generation trajectories. Motivated by this token error pattern, we propose CURE, a budget-aware dynamic repair tree designed to repair errors at uncertainty focal points without incurring prohibitive tree-verification overheads. Specifically, our method uses predictive confidence margins to dynamically locate candidate error tokens within a block-parallel draft, expands bounded repair paths only at these fragile nodes, and employs a novel repair resynchronization mechanism to realign draft states post-verification. Evaluations on code-generation benchmarks (HumanEval, MBPP, and LiveCodeBench-lite) and mathematical reasoning benchmark (GSM8K) demonstrate that CURE increases the average accepted length by 4.2-7.5% over parallel baselines without repair, translating to an end-to-end speedup of $2.66-3.49\times$ over target-only decoding. Furthermore, we provide a plug-and-play repair module compatible with standard parallel drafting frameworks. We also characterize the trade-off between draft compute and verification efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。