提出新方法提升扩散语言模型并行解码速度与稳定性。
Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decoding
- 通过缓存覆盖实现单次前向传播的稳定验证
- 减少90%以上无效重校验,解码速度提升3倍以上
- 适合需要高速且高质量生成的对话系统
并行扩散解码可通过每步解码多个标记加速扩散语言模型推理,但激进并行常损害质量。可撤销解码通过重新检查早期标记缓解此问题,但我们发现现有验证机制常引发翻转振荡——标记被重新遮蔽后又原样恢复。这种行为导致两个负面影响:遮蔽已验证位置削弱了并行起草的上下文条件,重复遮蔽循环消耗修订预算却无实质进展。我们提出COVER(缓存覆盖验证以实现高效修订),在单次前向传播中完成留一验证与稳定起草。COVER通过缓存覆盖构建两种注意力视图:选定种子标记被遮蔽用于验证,而其缓存的键值状态注入给其他所有查询以保留上下文信息,并采用闭式对角修正防止种子位置自泄漏。COVER还使用基于稳定性的评分策略,平衡不确定性、下游影响与缓存漂移,动态调整每步验证种子数量。在多个基准测试中,COVER显著减少不必要的修订,实现更快解码同时保持输出质量。
原文摘要 · Abstract (English)
Parallel diffusion decoding can accelerate diffusion language model inference by unmasking multiple tokens per step, but aggressive parallelism often harms quality. Revocable decoding mitigates this by rechecking earlier tokens, yet we observe that existing verification schemes frequently trigger flip-flop oscillations, where tokens are remasked and later restored unchanged. This behaviour slows inference in two ways: remasking verified positions weakens the conditioning context for parallel drafting, and repeated remask cycles consume the revision budget with little net progress. We propose COVER (Cache Override Verification for Efficient Revision), which performs leave-one-out verification and stable drafting within a single forward pass. COVER constructs two attention views via KV cache override: selected seeds are masked for verification, while their cached key value states are injected for all other queries to preserve contextual information, with a closed form diagonal correction preventing self leakage at the seed positions. COVER further prioritises seeds using a stability aware score that balances uncertainty, downstream influence, and cache drift, and it adapts the number of verified seeds per step. Across benchmarks, COVER markedly reduces unnecessary revisions and yields faster decoding while preserving output quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。