用离散扩散模型实现高保真图像超分辨率,解决稀有纹理和空间不一致问题。
Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution

- 通过逆频采样强化罕见但关键的视觉纹理
- 仅需几步并行解码即达优秀感知质量
- 适合追求高效高质图像生成的研究者
连续扩散模型已成为高保真图像超分辨率(SR)的主流范式,但通常将重建建模为连续信号去噪,并通过外部条件模块引入语义先验,难以直接利用现代多模态模型的统一标记化缩放机制。自回归模型虽以离散视觉标记表示图像,但因果解码效率低。离散扩散在标记上提供非因果并行预测,是潜在折中方案。然而,直接应用于SR面临两大挑战:(1)视觉标记的长尾分布导致稀有但感知关键纹理被低估;(2)并行解码空间不一致,易引入孤立伪影。为此,我们提出DiMOO-SR,一种面向真实图像超分辨率的稀有性感知多模态离散扩散框架。训练时采用逆频采样(IFS)优先处理低频但信息丰富的标记;推理时使用空间一致性排序(SCR)基于局部邻域一致性重估标记置信度,提升结构连贯性。在多个真实世界SR基准测试中,DiMOO-SR仅需少数并行解码步骤即可达到竞争性感知质量,凸显了离散扩散在生成式图像超分辨率中的潜力。代码将在发表后公开。
原文摘要 · Abstract (English)
Continuous diffusion models have become the dominant paradigm for photo-realistic image Super-Resolution (SR), but they typically formulate reconstruction as continuous signal-level denoising and incorporate semantic priors through external conditioning modules. This makes it less direct to exploit the unified token-based scaling paradigm of modern multimodal models. Autoregressive models provide a more native semantic representation by modeling images as discrete visual tokens, yet their causal decoding is inefficient for high-resolution reconstruction. Discrete diffusion offers a promising middle ground by enabling non-causal, parallel prediction over visual tokens. However, directly adapting discrete diffusion to SR remains non-trivial due to two task-specific challenges: (1) the long-tailed distribution of visual tokens, which under-represents rare but perceptually critical textures; and (2) spatially inconsistent parallel decoding, which may introduce isolated artifacts. To address these issues, we propose DiMOO-SR, a rarity-aware multimodal discrete diffusion framework for photo-realistic image SR. During training, Inverse Frequency Sampling (IFS) prioritizes under-represented but information-rich tokens. During inference, Spatial Consistency Ranking (SCR) refines token confidence using local neighborhood agreement to improve structural coherence. Extensive experiments on widely used real-world SR benchmarks demonstrate that DiMOO-SR achieves competitive perceptual quality with only a few parallel decoding steps, highlighting the potential of discrete diffusion for generative image super-resolution. The code will be released upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。