通过置信度与持续性信号,提前停止无需重做的文本生成步骤。
CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference
- 基于原始模型输出的置信度和稳定性判断是否继续迭代。
- 在GSM8K和HumanEval上速度提升2.7到3.3倍,准确率仅下降1.22点。
- 无需训练或修改模型,适用于多种推理场景,适合高效生成应用。
扩散语言模型(DLM)并行更新多个词元,但实际解码常采用固定去噪时长。许多预测早期已稳定,但块式解码仍持续至所有位置完成,导致重复密集前向计算。现有加速方法多依赖学习过滤器、修改得分或缓存机制。本文提出CORADiff,一种无需训练的方法,仅对未被原规则解决的位置应用置信度与持续性门控。已接受词元保持可见作为上下文,块在所有位置解决后终止。该方法不需修改主干模型、学习接受模型或调整logits。理论解释高置信度且稳定的预测更可能匹配固定时长密集解码终点,实验验证了干预前后轨迹的一致性。在独立的GSM8K校准子集上选定一个运行点并冻结用于所有评估。在匹配的Learn2PD风格LLaDA协议下,CORADiff在八种任务长度设置中运行时间最低。五种情况下任务得分持平或超过密集解码,最大下降仅1.22分。相比EOS感知密集解码,其在GSM8K和HumanEval上的增量加速分别达2.70x和3.32x。在固定时长1024/1024机制隔离协议下达到13.14x加速,并在无需调参的情况下迁移至Dream,实现3.18x–3.53x加速。结果表明,原生置信度与持续性信号可实现可靠残差接受,减少重复去噪计算同时保持任务质量。
原文摘要 · Abstract (English)
Diffusion language models (DLMs) update many tokens in parallel, yet practical decoders often use a fixed denoising horizon. Many predictions stabilize early, but blockwise decoding continues until all positions are resolved, causing repeated dense forward passes. Existing accelerators often rely on learned filters, modified scores, dependency models, or cache-specific mechanisms. We ask whether native trajectory signals can identify residual positions likely to match the deterministic dense endpoint. We propose CORA-Diff, a training-free method that preserves the original transfer rule and applies confidence-and-persistence gating only to positions that rule leaves unresolved. Accepted tokens remain visible as context, and the block terminates once all positions are resolved. This requires no backbone change, learned acceptance model, or logit modification. Our theory explains why high-confidence, persistent predictions are more likely to match the fixed-horizon dense endpoint, and paired post-intervention trajectories provide direct empirical support. We select one operating point on a separate GSM8K calibration subset and freeze it for all evaluations. Under a matched Learn2PD-style LLaDA protocol, CORA-Diff has the lowest measured runtime in all eight task-length settings. Task scores match or exceed dense decoding in five settings, and the largest observed drop is 1.22 points. Its incremental speedups over EOS-aware dense decoding are 2.70x and 3.32x on GSM8K and HumanEval. It also reaches 13.14x under the fixed-horizon 1024/1024 mechanism-isolation protocol and transfers to Dream without retuning at 3.18x-3.53x. These results show that native confidence and persistence enable reliable residual acceptance, reducing repeated denoising computation while preserving task quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。