arXiv:2410.21535cs.CV2024-10NeurIPS被引 26

用Retinex理论和Mamba结构提升图像曝光校正效率与效果

ECMamba: Consolidating Selective State Space Model with Retinex Guidance for Efficient Multiple Exposure Correction

  • 分路径处理反射率与光照图,结合Retinex理论指导恢复
  • 引入新型2D可变形扫描状态空间层,提升计算效率与精度
  • 适合需要高效高质曝光修复的低级视觉任务研究者

曝光校正(EC)旨在恢复过曝或欠曝图像的正确曝光条件。现有深度学习模型虽表现良好,但极少将Retinex理论融入架构设计,方法论存在空白。同时,性能与效率之间的平衡仍是未充分探索的问题。受Mamba在序列建模中高效性的启发,本文提出基于Mamba的新型曝光校正框架ECMamba,采用双路径设计,分别用于恢复反射率与光照图。首先推导Retinex理论,并训练一个能够将输入映射到两个中间空间的Retinex估计器,分别逼近目标反射率与光照图,以促进后续曝光校正模块的精细化重建。此外,我们设计了基于Retinex信息引导的新型2D选择性状态空间层(Retinex-SS2D),作为核心运算单元。该结构采用创新的2D可变形特征聚合扫描策略,在保证性能的同时显著提升效率。大量实验结果与详尽消融研究验证了所提ECMamba的卓越性能及各组件的重要性。代码已开源:https://github.com/LowlevelAI/ECMamba。

原文摘要 · Abstract (English)

Exposure Correction (EC) aims to recover proper exposure conditions for images captured under over-exposure or under-exposure scenarios. While existing deep learning models have shown promising results, few have fully embedded Retinex theory into their architecture, highlighting a gap in current methodologies. Additionally, the balance between high performance and efficiency remains an under-explored problem for exposure correction task. Inspired by Mamba which demonstrates powerful and highly efficient sequence modeling, we introduce a novel framework based on Mamba for Exposure Correction (ECMamba) with dual pathways, each dedicated to the restoration of reflectance and illumination map, respectively. Specifically, we firstly derive the Retinex theory and we train a Retinex estimator capable of mapping inputs into two intermediary spaces, each approximating the target reflectance and illumination map, respectively. This setup facilitates the refined restoration process of the subsequent Exposure Correction Mamba Module (ECMM). Moreover, we develop a novel 2D Selective State-space layer guided by Retinex information (Retinex-SS2D) as the core operator of ECMM. This architecture incorporates an innovative 2D scanning strategy based on deformable feature aggregation, thereby enhancing both efficiency and effectiveness. Extensive experiment results and comprehensive ablation studies demonstrate the outstanding performance and the importance of each component of our proposed ECMamba. Code is available at https://github.com/LowlevelAI/ECMamba.

图像修复MambaRetinex曝光校正

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。