arXiv:2608.16793cs.CV2026-08

用像素空间扩散变换器实现高效统一图像修复,不依赖预训练模型。

PixRestore: Unified Image Restoration via Pixel Diffusion Transformer

论文配图:PixRestore: Unified Image Restoration via Pixel Diffusion Transformer
图 1 · 摘自论文原文
  • 直接在像素块上做流匹配,避免潜在细节丢失。
  • 仅5000万参数,单步推理,综合质量领先同类方法。
  • 适合追求高效率与多退化类型修复的实用场景。

统一图像修复(UIR)旨在使用单一模型从具有不同退化的低质量(LQ)图像中恢复高质量(HQ)内容。现有方法通常基于大型预训练文本到图像(T2I)潜空间扩散模型,但其变分自编码器(VAE)可能丢弃修复敏感细节,且开放生成先验会引入内容不一致伪影。本文提出PixRestore,一种无VAE的像素空间扩散变换器(DiT),其扩散主干完全从零训练,不依赖T2I预训练。PixRestore在分块像素上直接执行流匹配,保留细粒度细节的同时保持序列可处理性。为适应不同退化,通过LQ-HQ DINO特征相似性学习层特征可靠性,更可靠层融合为密集条件,不可靠层则获得更强的HQ特征监督以促进退化消除。我们在大规模多样场景与退化数据集上训练PixRestore,并利用DINO-based对抗目标微调为单步生成器以实现高效推理。在公开基准和真实世界测试集上的实验表明,仅约5000万参数、单步推理下,PixRestore在整体保真度、感知质量和退化鲁棒性上均优于现有UIR模型,且效率显著更高。更大版本进一步提升性能,证明了像素空间设计的可扩展性。代码与构建的基准见https://github.com/csslc/PixRestore。

原文摘要 · Abstract (English)

Unified image restoration (UIR) aims to recover high-quality (HQ) content from low-quality (LQ) images with different degradations using a single model. Most recent methods adapt large pretrained text-to-image (T2I) latent diffusion models for their strong capacity and generative priors. However, the variational autoencoder (VAE) in latent T2I models may discard restoration-sensitive details, while the open-ended synthesis prior can introduce content-inconsistent artifacts. We present PixRestore, a VAE-free pixel-space Diffusion Transformer (DiT) for UIR, where the diffusion backbone is trained entirely from scratch, without relying on T2I pretraining. PixRestore performs flow matching directly on patchified pixels, preserving fine-grained details while keeping the token sequence tractable. To adapt to different degradations, PixRestore learns to predict the reliability of layer features using LQ--HQ DINO feature similarity. Features from more reliable layers are fused as dense conditioning, while less reliable layers receive stronger HQ-feature supervision to encourage degradation removal. We train PixRestore on a large-scale corpus of diverse scenes and degradations, and further finetune it into a one-step generator using DINO-based adversarial objectives for efficient inference. Experiments on public benchmarks and real-world test sets show that, with only about 50M parameters and single-step inference, PixRestore achieves the best overall fidelity, perceptual quality, and robustness to degradations among competing UIR models while being far more efficient. Larger PixRestore variants can further boost performance, demonstrating the scalability of our pixel-space design. Code and the curated benchmark can be found at https://github.com/csslc/PixRestore.

图像修复扩散模型像素空间高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。