通过动态切换搜索与加速，提升扩散语言模型的生成质量与效率。
Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models
- 根据模型置信度动态调整解码策略：低信心时拓宽搜索，高信心时并行解码
- 在GSM8K、MBPP等任务上显著提升生成质量，推理速度保持竞争力
- 无需训练即可部署，适合需要高质量文本生成的场景
扩散语言模型（DLMs）通过迭代去噪掩码序列生成文本，每一步需决定哪些位置进行解码。标准解码采用贪心策略：选择置信度最高的位置解码，但这种局部决策可能导致次优解码顺序，尤其在推理类提示中表现不佳。本文提出SOAR，一种无需训练的解码算法，可根据模型不确定性自适应调整行为：当置信度低时，短暂扩展对替代解码顺序的搜索以避免过早确定；当置信度高时，则收缩搜索范围，批量并行解码多个位置，减少去噪迭代次数。在Dream-7B和LLaDA-8B模型上，于数学推理（GSM8K）与代码生成（MBPP、HumanEval）基准测试中，SOAR在提升生成质量的同时保持了有竞争力的推理速度，为扩散语言模型的解码提供了兼顾质量与效率的实用方案。代码已开源。
原文摘要 · Abstract (English)
Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, repeatedly deciding which positions to commit at each step. Standard decoding follows a greedy rule: unmask the most confident positions, yet this local choice can lock the model into a suboptimal unmasking order, especially on reasoning-heavy prompts. We present SOAR, a training-free decoding algorithm that adapts its behavior to the model's uncertainty. When confidence is low, SOAR briefly widens the search over alternative unmasking decisions to avoid premature commitments; when confidence is high, it collapses the search and decodes many positions in parallel to reduce the number of denoising iterations. Across mathematical reasoning and code generation benchmarks (GSM8K, MBPP, HumanEval) on Dream-7B and LLaDA-8B, SOAR improves generation quality while maintaining competitive inference speed, offering a practical way to balance quality and efficiency in DLM decoding. Our Code is available at https://github.com/duterscmy/SOAR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。