解决单图反射去除中特征语义差异与标签不一致问题
GFRRN: Explore the Gaps in Single Image Reflection Removal
- 用可学习的Mona层实现预训练模型与去反射模型间的高效对齐
- 设计标签生成器统一合成与真实数据的反射标签，提升泛化能力
- 引入自适应频率学习与动态注意力机制，增强细节保留与结构建模
先前基于双流结构并结合特征交互的方法在单图反射去除（SIRR）任务中表现优异，但仍面临两大挑战：(1) 预训练模型特征与去反射模型特征之间的语义理解差距；(2) 合成数据与真实世界数据间反射标签的不一致性。本文首次采用参数高效微调（PEFT）策略，通过在预训练模型中集成多个可学习的Mona层以对齐训练方向。进一步提出标签生成器，统一合成与真实数据的反射标签。此外，设计基于高斯分布的自适应频率学习模块（G-AFLB），用于自适应学习和融合频率先验；采用动态代理注意力（DAA）替代传统窗式注意力，动态建模窗口间（inter-）与窗内（intra-）的重要性水平。上述组件构成提出的无间隙反射去除网络（GFRRN）。大量实验表明，该方法显著优于现有SIRR先进方法。
原文摘要 · Abstract (English)
Prior dual-stream methods with the feature interaction mechanism have achieved remarkable performance in single image reflection removal (SIRR). However, they often struggle with (1) semantic understanding gap between the features of pre-trained models and those of reflection removal models, and (2) reflection label inconsistencies between synthetic and real-world training data. In this work, we first adopt the parameter efficient fine-tuning (PEFT) strategy by integrating several learnable Mona layers into the pre-trained model to align the training directions. Then, a label generator is designed to unify the reflection labels for both synthetic and real-world data. In addition, a Gaussian-based Adaptive Frequency Learning Block (G-AFLB) is proposed to adaptively learn and fuse the frequency priors, and a Dynamic Agent Attention (DAA) is employed as an alternative to window-based attention by dynamically modeling the significance levels across windows (inter-) and within an individual window (intra-). These components constitute our proposed Gap-Free Reflection Removal Network (GFRRN). Extensive experiments demonstrate the effectiveness of our GFRRN, achieving superior performance against state-of-the-art SIRR methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。