通过自适应掩码实现图像修复的鲁棒表征学习,兼顾各类退化场景。
RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration
- 采用像素级掩码聚焦语义丰富区域,联合学习生成先验与内容先验。
- 在多种退化条件下均达顶尖性能,尤其在极端和混合退化下表现突出。
- 适合需要高泛化能力的图像修复任务,如真实世界图像恢复。
本文提出一种名为RAM++的两阶段全功能图像修复框架,通过将高层语义理解与低层纹理生成结合,实现面向内容的鲁棒修复。针对现有退化导向方法在极端场景(如退化与图像结构强耦合)下的不足,RAM++通过三项核心设计克服常见挑战:1)自适应语义感知掩码(AdaSAM):预训练阶段对语义丰富区域施加像素级掩码,使网络从各类退化中同时学习生成先验与内容先验;2)掩码属性导通(MAC):选择性微调策略,调节高贡献层以弥合掩码预训练与全图微调间的完整性差距,同时保留已学先验;3)鲁棒特征正则化(RFR):利用DINOv2的语义一致且退化不变表示,结合高效特征融合,实现忠实且语义连贯的修复。实验表明,RAM++在已见、未见、极端及混合退化场景下均取得稳健、均衡且领先的性能。代码与模型将公开于https://github.com/DragonisCV/RAM。
原文摘要 · Abstract (English)
This work presents Robust Representation Learning via Adaptive Mask (RAM++), a two-stage framework for all-in-one image restoration. RAM++ integrates high-level semantic understanding with low-level texture generation to achieve content-oriented robust restoration. It addresses the limitations of existing degradation-oriented methods in extreme scenarios (e.g., degradations strongly coupled with image structures). RAM++ also mitigates common challenges such as unbalanced performance across tasks, overfitting to seen degradations, and weak generalization to unseen ones through three key designs: 1) Adaptive Semantic-Aware Mask (AdaSAM): a pretraining strategy that applies pixel-level masks to semantically rich and textured regions. This design enables the network to learn both generative priors and image content priors from various degradations. 2) Mask Attribute Conductance (MAC): a selective fine-tuning strategy that adjusts the layers with higher contributions to bridge the integrity gap between masked pretraining and full-image fine-tuning while retaining learned priors. 3) Robust Feature Regularization (RFR): a strategy that leverages DINOv2's semantically consistent and degradation-invariant representations, together with efficient feature fusion, to achieve faithful and semantically coherent restoration. With these designs, RAM++ achieves robust, well-balanced, and state-of-the-art performance across seen, unseen, extreme, and mixed degradations. Our code and model will be released at https://github.com/DragonisCV/RAM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。