arXiv:2606.29814cs.CV2026-06

改进离散扩散模型,实现高分辨率图像生成的精准与高效。

Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis

论文配图:Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis
图 1 · 摘自论文原文
  • 引入动态编辑机制,可迭代修正已解码图像区块。
  • 在GenEval达0.90分,DPG为86.9,HPSv3为10.76。
  • 适合关注图像生成质量与训练效率的研究者。

我们提出Nemotron-Labs-Diffusion-Image,一种先进的掩码离散扩散模型(MDM),用于高分辨率文本到图像合成。相较于以往的掩码图像生成方法,该模型解决了两大挑战:首先,标准离散扩散模型缺乏自校正能力,因离散标记一旦解码便无法修改;其次,增大离散图像标记器的词表规模虽提升重建保真度,但导致生成建模的训练信号愈发稀疏。为此,我们设计了一种标记编辑机制,在推理阶段动态修正已解码标记,类似雕塑家反复打磨作品;同时提出分组交叉熵(GCE)损失函数,为嵌入空间中邻近真实标记的候选标记赋予正向学习信号,缓解信号稀疏问题。此外,针对大词表场景,我们实现一种定制化融合算子,显著降低显存占用。实验表明,这些创新大幅提升了训练效率与图像保真度,在GenEval上得分0.90,DPG为86.9,HPSv3为10.76。

原文摘要 · Abstract (English)

We propose Nemotron-Labs-Diffusion-Image, a state-of-the-art masked discrete diffusion model (MDM) for high-resolution text-to-image synthesis. Compared with prior work on masked image generation, Nemotron-Labs-Diffusion-Image addresses two key challenges. First, unlike continuous diffusion models which progressively refine latent representations across the entire image, standard MDMs lack self-correcting capability because discrete tokens cannot be modified once they are unmasked. Second, although increasing the vocabulary size of discrete image tokenizers improves reconstruction fidelity, it introduces optimization difficulties for generative modeling as the per-token training signal becomes increasingly sparse. To address the first challenge, Nemotron-Labs-Diffusion-Image incorporates a token-editing mechanism that enables the model to dynamically revise already-unmasked tokens during inference, similar to how a sculptor iteratively refines their work. To tackle the second challenge, we propose a Grouped Cross-Entropy (GCE) objective that assigns positive learning signals to tokens neighboring the ground truth in embedding space, thereby alleviating signal sparsity. To further improve training efficiency, we implement a custom fused operator for GCE that significantly reduces VRAM usage in large-vocabulary settings. Experimental results demonstrate that these innovations substantially improve both training efficiency and image fidelity of masked discrete image generators, achieving a score of 0.90 on GenEval, 86.9 on DPG and 10.76 of HPSv3.

图像生成扩散模型离散扩散高质量合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。