首个多视角暗光图像增强框架,通过协同特征融合提升不同视角的还原效果。
RCNet: Deep Recurrent Collaborative Network for Multi-View Low-Light Image Enhancement
- 设计递归协同模块,分步实现视图内增强与视图间对齐融合
- 在1860组三视图数据上性能超越现有方法,显著改善暗光场景还原
- 适用于多相机系统下的夜间视觉增强,适合计算机视觉研究者
多视角场景观察可带来更全面的视觉体验,但在暗光条件下,高度相关的多视图图像却严重失真,影响辅助视角的场景理解。现有单图增强方法因忽略多视图间潜在特征对应关系,难以在所有视角上保持一致的高质量恢复。为此,我们首次探索多视角暗光图像增强任务。首先构建新数据集Multi-View Low-light Triplets(MVLT),包含1860对三视图图像,覆盖大光照范围和广噪声分布。每对三视图拍摄同一场景的不同视角。其次提出基于递归协同网络(RCNet)的深度多视图增强框架。核心为递归特征增强、对齐与融合(ReEAF)模块:先进行视图内特征增强(Intra-view EN),再通过视图间特征对齐与融合(Inter-view AF)建模跨视图特征传播。同时设计从增强到对齐(E2A)与从对齐到增强(A2E)两个交互模块,分别利用注意力加权和采样策略促进视图内与视图间信息交互。实验表明,所提方法显著优于当前最优方法。所有数据集、代码与模型将公开于https://github.com/hluo29/RCNet。
原文摘要 · Abstract (English)
Scene observation from multiple perspectives would bring a more comprehensive visual experience. However, in the context of acquiring multiple views in the dark, the highly correlated views are seriously alienated, making it challenging to improve scene understanding with auxiliary views. Recent single image-based enhancement methods may not be able to provide consistently desirable restoration performance for all views due to the ignorance of potential feature correspondence among different views. To alleviate this issue, we make the first attempt to investigate multi-view low-light image enhancement. First, we construct a new dataset called Multi-View Low-light Triplets (MVLT), including 1,860 pairs of triple images with large illumination ranges and wide noise distribution. Each triplet is equipped with three different viewpoints towards the same scene. Second, we propose a deep multi-view enhancement framework based on the Recurrent Collaborative Network (RCNet). Specifically, in order to benefit from similar texture correspondence across different views, we design the recurrent feature enhancement, alignment and fusion (ReEAF) module, in which intra-view feature enhancement (Intra-view EN) followed by inter-view feature alignment and fusion (Inter-view AF) is performed to model the intra-view and inter-view feature propagation sequentially via multi-view collaboration. In addition, two different modules from enhancement to alignment (E2A) and from alignment to enhancement (A2E) are developed to enable the interactions between Intra-view EN and Inter-view AF, which explicitly utilize attentive feature weighting and sampling for enhancement and alignment, respectively. Experimental results demonstrate that our RCNet significantly outperforms other state-of-the-art methods. All of our dataset, code, and model will be available at https://github.com/hluo29/RCNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。