通过连续空间优化，让扩散语言模型推理更快更准。
Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference
- 引入连续混合状态，在离散解码中迭代优化词元表示。
- 实现2到8倍加速，且不损失生成质量。
- 无需训练，适合追求高效推理的研究者和开发者。
扩散大语言模型（DLLMs）虽能实现快速的非自回归推理，但在并行解码时面临严重的质量-速度权衡，根源在于‘组合矛盾’现象——并行生成的词元会形成语义不一致的组合。为此，本文将连续表示引入离散解码过程，以保留词元间的丰富位置依赖关系。提出ReMix（拒绝混合）框架，构建一个介于初始掩码状态与最终解码状态之间的连续混合状态。该中间状态使词元表示在连续空间中迭代精炼，解决与其他词元的相互冲突后再坍缩为离散样本。此外，通过拒绝规则将不确定的连续表示回退至掩码状态重新处理，确保稳定性并防止错误传播。实验表明，ReMix作为无训练方法，在不降低生成质量的前提下，实现了2至8倍的推理加速。
原文摘要 · Abstract (English)
Diffusion Large Language Models (DLLMs) promise fast non-autoregressive inference but suffer a severe quality-speed trade-off in parallel decoding. This stems from the ''combinatorial contradiction'' phenomenon, where parallel tokens form semantically inconsistent combinations. We address this by integrating continuous representations into the discrete decoding process, as they preserve rich inter-position dependency. We propose ReMix (Rejection Mixing), a framework that introduces a novel Continuous Mixing State as an intermediate between the initial masked state and the final decoded token state. This intermediate state allows a token's representation to be iteratively refined in a continuous space, resolving mutual conflicts with other tokens before collapsing into a final discrete sample. Furthermore, a rejection rule reverts uncertain representations from the continuous state back to the masked state for reprocessing, ensuring stability and preventing error propagation. ReMix thus mitigates combinatorial contradictions by enabling continuous-space refinement during discrete diffusion decoding. Extensive experiments demonstrate that ReMix, as a training-free method, achieves a $2-8 \times$ inference speedup without any quality degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。