根据图像修复难度动态选择网络,提升真实世界超分效率与效果
Efficient Difficulty-Aware Dynamic Routing for Diffusion-Based Real-World Image Super-Resolution

- 通过难度估计算法为不同图像分配合适容量的网络
- 采用可调VAE下采样率,保留高细节信息,提升难修复图像质量
- 兼顾效率与精度,适合追求快速高质量超分的应用场景
基于扩散模型的图像超分辨率方法利用预训练的Stable Diffusion(SD)模型作为强大的生成先验,取得了显著进展。然而,现有方法仍存在两大瓶颈:一是单步或多步方法对所有输入采用统一处理策略,忽略图像间修复难度差异;二是SD模型中VAE的激进下采样(如8倍)导致高频细节不可逆丢失,后续扩散过程无法恢复。为此,本文提出难度感知动态路由(DDR)策略,打破固定处理范式。首先设计难度估计算法预测每张图像的修复成本,实现自动分配合适容量的网络;其次通过调节SD主干中VAE的空间下采样率,构建一系列容量不同的Real-ISR网络,在复杂图像上保留更多高频信息,同时保持简单输入的处理效率。大量实验表明,该模型在速度与性能上均优于当前最先进方法。
原文摘要 · Abstract (English)
Diffusion-based methods have achieved impressive performance in real-world image super-resolution (Real-ISR) by leveraging large pre-trained stable diffusion (SD) models as powerful generative priors. However, these methods still face two key limitations. First, existing SD-based one-step and multi-step Real-ISR approaches adopt a unified processing paradigm for all input samples, ignoring the varying restoration difficulty across images. Second, the aggressive resolution reduction of the VAE in SD models (e.g., 8x downsampling) leads to irreversible loss of fine-scale details, which cannot be recovered by the subsequent diffusion process. To address these limitations, we propose a Difficulty-aware Dynamic Routing (DDR) strategy that overcomes the rigid, one-size-fits-all processing paradigm. Specifically, we first design a difficulty estimator to predict the restoration cost of each input image, enabling automatic assignment to a network of appropriate capacity. Then, we construct a set of Real-ISR networks with varying model capacities by modulating the spatial downsampling ratio of the VAE in the SD backbone, thereby preserving more high-frequency information for challenging cases while maintaining efficiency for simpler inputs. Extensive experiments have demonstrated the superior efficiency and effectiveness of the proposed model compared to recent state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。