UHDformer++用四空间协同提升超高清图像修复,参数减少86%仍保持领先性能。
Ultra-High-Definition Restoration Transformers with Correlation Matching Transformation

- 构建高/低分辨率与超分融合空间,实现多尺度特征协同重建。
- 通过特征精炼匹配和自适应通道调制,提升低分辨率特征质量。
- 在5类超高清修复任务中表现优异,适合资源受限的图像增强场景。
我们提出UHDformer++,一种通用的基于Transformer的框架,用于解决多种超高清(UHD)图像修复任务。该框架在4个协同学习空间运行:1)高分辨率空间(HR),用于多层次特征提取;2)低分辨率空间(LR),用于学习紧凑且具代表性的特征;3)超分辨率空间(SR),用于将低分辨率特征上采样;4)低-高融合与重建空间(LHFR),用于最终图像恢复。其中,HR提取多尺度高分辨率特征,并与低分辨率线索融合生成残差图;LR从HR中提炼互补表征以提升修复质量。为丰富LHFR特征,先对LR输出进行超分处理。我们引入两个模块连接高低分辨率空间:特征精炼相关匹配变换(FR-CMT)模块从最大值与均值池化后的高分辨率特征融合结果中选取前C/r个通道(C为通道数,r≥1控制压缩程度),替换低分辨率Transformer中信息量较少的通道;自适应通道调制(ACM)模块自适应重校准多尺度高分辨率特征,确保仅任务相关的信息传递至低分辨率空间。大量实验表明,相较于最新方法,UHDformer++模型参数减少至少86%,同时在5项超高清修复任务(包括低光增强、去雾、去模糊、去雨、去雪)中均取得显著性能提升。代码将发布于https://github.com/supersupercong/uhdformerplus。
原文摘要 · Abstract (English)
We propose UHDformer++, a general Transformer-based framework to solve numerous Ultra-High-Definition (UHD) image restoration tasks. UHDformer++ operates across $4$ coordinated learning spaces: 1) a high-resolution space (HR) for multi-level feature extraction, 2) a low-resolution space (LR) for learning compact, representative features, 3) a super-resolution space (SR) for upsampling low-resolution features from SR, and 4) a low-high fusion and reconstruction space (LHFR) for final image restoration. Specifically, HR extracts multi-scale high-resolution features and fuses them with low-resolution cues to produce residual images, while LR distills complementary representations from HR to improve restoration quality. To supply LHFR with richer features, SR super-resolves LR outputs before fusion. We further introduce two modules to bridge the high- and low-resolution spaces. The Feature-Refined Correlation Matching Transformation (FR-CMT) module selects the top $C/r~(C~\text{denotes the number of channels;~}r\geq1~\text{controls the squeezing level})$, from the fusion between max- and mean-pooled high-resolution features to replace less informative channels in the low-resolution Transformer. The Adaptive Channel Modulator (ACM) adaptively recalibrates multi-scale high-resolution features, ensuring that only task-relevant information propagates to LR. Extensive experiments demonstrate that UHDformer++ reduces model parameters by at least 86\% compared with recent state-of-the-art methods while achieving substantial performance gains across $5$ UHD restoration tasks, including low-light image enhancement, dehazing, deblurring, deraining, and desnowing. Code will be released at https://github.com/supersupercong/uhdformerplus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。