提出预见性解码方法，提升大语言扩散模型的生成效率与质量。
Decoding Large Language Diffusion Models with Foreseeing Movement
- 采用搜索策略融合局部与全局信息，优化离散空间中的解码顺序。
- 在多个基准上实现更优的效率-性能平衡，加速效果显著。
- 适合追求高效高质文本生成的研究者与开发者使用。
大语言扩散模型（LLDMs）得益于灵活的解码机制，可实现并行推理和可控生成，优于自回归模型。然而这种灵活性也带来挑战：推理性能对词元解码顺序高度敏感。现有启发式方法多关注局部影响，忽视长期后果。为此，本文提出预见性解码方法（FDM），通过搜索策略结合局部与全局考量，有效优化离散空间中的解码路径。进一步地，基于完整解码过程中所选词元的一致性分析，我们设计了加速变体FDM-A，仅在关键步骤进行深度探索，兼顾探索与平衡。在多种基准和模型架构上的实验验证了FDM的可扩展性，并表明FDM-A在效率与性能之间取得更优权衡。本工作或为构建更强大的LLDM解码方法提供原则性思路。
原文摘要 · Abstract (English)
Large Language Diffusion Models (LLDMs) benefit from a flexible decoding mechanism that enables parallelized inference and controllable generations over autoregressive models. Yet such flexibility introduces a critical challenge: inference performance becomes highly sensitive to the decoding order of tokens. Existing heuristic methods, however, focus mainly on local effects while overlooking long-term impacts. To address this limitation, we propose the Foreseeing Decoding Method (FDM), a novel approach that integrates both local and global considerations to unlock the full potential, employing a search-based strategy to enable effective optimization in discrete spaces. Furthermore, by analyzing the consistency of chosen tokens in the full decoding process, we develop a variant, FDM with Acceleration (FDM-A), which restricts deep exploration to critical steps identified as the exploration and balance circumantences. Extensive experiments across diverse benchmarks and model architectures validate the scalability of FDM and demonstrate the superior efficiency-performance trade-off achieved by FDM-A. Our work might potentially provide a principled step toward more powerful decoding methods for LLDMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。