通过联合搜索生成顺序与词元空间，提升扩散语言模型的解码效果。
Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
- 同时优化生成顺序和词元选择，拓展解码路径探索空间。
- 在GSM8K等4个基准上分别提升3.1%~7.9%，优于基线模型。
- 适合追求高精度生成的数学推理与代码生成任务使用。
扩散语言模型（DLMs）支持无序生成，可探索多种解码路径。但现有方法仅固定单一路径，限制了轨迹空间的探索。本文提出“顺序-词元联合搜索”，通过联合搜索生成顺序与词元值，实现对轨迹空间的高效探索。其核心是一个用于评估去噪动作的似然估计器，支持稳定剪枝与多样化路径探索。在数学推理与代码生成多个基准测试中，该方法在GSM8K、MATH500、Countdown和HumanEval上分别实现3.1%、3.8%、7.9%和6.8%的绝对性能提升，超越基线模型，并达到或超过微调后的diffu-GRPO post-trained d1-LLaDA表现。本工作确立了联合搜索在推进DLM解码中的关键作用。
原文摘要 · Abstract (English)
Diffusion Language Models (DLMs) offer order-agnostic generation that can explore many possible decoding trajectories. However, current decoding methods commit to a single trajectory, limiting exploration in trajectory space. We introduce Order-Token Search to explore this space through jointly searching over generation order and token values. Its core is a likelihood estimator that scores denoising actions, enabling stable pruning and efficient exploration of diverse trajectories. Across mathematical reasoning and coding benchmarks, Order-Token Search consistently outperforms baselines on GSM8K, MATH500, Countdown, and HumanEval (3.1%, 3.8%, 7.9%, and 6.8% absolute over backbone), matching or surpassing diffu-GRPO post-trained d1-LLaDA. Our work establishes joint search as a key component for advancing decoding in DLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。