提出新模型平衡理解与生成，显著提升文本图像生成效果。
Balancing Understanding and Generation in Discrete Diffusion Models
- 用平稳噪声核统一两类扩散模型，理论上有据可依。
- 零样本文本任务超UDLM 5.4分，32步生成FID达54.1。
- 适合追求生成质量与理解能力均衡的研究者使用。
在离散生成建模中，掩码扩散语言模型（MDLM）擅长语义理解与零样本泛化，而均匀噪声扩散语言模型（UDLM）在少步生成上表现优异，但两者均难以兼顾两方面性能。为此，本文提出XDLM，通过平稳噪声核融合两类范式。其核心贡献为：（1）从理论上统一MDLM与UDLM，使其成为特例；（2）通过后验概率的代数简化缓解内存瓶颈。实验表明，XDLM在理解能力与生成质量间实现更优权衡。定量结果：在零样本文本任务上比UDLM高5.4分，少步图像生成FID为54.1，优于MDLM的80.8。当扩展至80亿参数大模型时，32步内实现15.0 MBPP，性能近乎翻倍。训练动态分析显示其具备更强长期扩展潜力。代码已开源。
原文摘要 · Abstract (English)
In discrete generative modeling, two dominant paradigms demonstrate divergent capabilities: Masked Diffusion Language Models (MDLM) excel at semantic understanding and zero-shot generalization, whereas Uniform-noise Diffusion Language Models (UDLM) achieve strong few-step generation quality, yet neither attains balanced performance across both dimensions. To address this, we propose XDLM, which bridges the two paradigms via a stationary noise kernel. XDLM offers two key contributions: (1) it provides a principled theoretical unification of MDLM and UDLM, recovering each paradigm as a special case; and (2) an alleviated memory bottleneck enabled by an algebraic simplification of the posterior probabilities. Experiments demonstrate that XDLM advances the Pareto frontier between understanding capability and generation quality. Quantitatively, XDLM surpasses UDLM by 5.4 points on zero-shot text benchmarks and outperforms MDLM in few-step image generation (FID 54.1 vs. 80.8). When scaled to tune an 8B-parameter large language model, XDLM achieves 15.0 MBPP in just 32 steps, effectively doubling the baseline performance. Finally, analysis of training dynamics reveals XDLM's superior potential for long-term scaling. Code is available at https://github.com/MzeroMiko/XDLM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。