提升复杂光照下偏振图像融合效果,兼顾全局与细节。
A Luminance-Aware Multi-Scale Network for Polarization Image Fusion with a Multi-Scene Dataset

- 设计亮度感知多尺度网络,动态融合亮度信息。
- 在多个数据集上显著优于现有方法,指标提升超50%。
- 适合图像融合、缺陷检测等需要高精度表面分析的场景。
偏振图像融合通过结合S0与DOLP图像,揭示表面粗糙度和材料特性,在伪装识别、组织病理分析、表面缺陷检测等领域有重要应用。为在复杂光照环境下有效整合不同偏振图像的互补信息,本文提出亮度感知多尺度网络(MLSN)。编码阶段引入亮度分支生成多尺度空间权重矩阵,动态将亮度信息注入特征图,解决偏振图像固有的对比度差异问题。瓶颈层设计全局-局部特征融合机制,通过窗口自注意力计算,结合残差连接实现特征维度重构,平衡全局上下文与局部细节。解码阶段提出亮度增强模块,建立亮度分布与纹理特征间的映射关系,实现融合结果的非线性亮度校正。此外,构建MSP数据集,包含1000对覆盖17类室内外复杂光照场景的偏振原始图像,提供四向偏振图。在MSP、PIF和GAND数据集上的大量实验表明,所提MLSN在主观与客观评价中均优于当前最优方法,各指标平均提升8.57%至63.53%,其中MS-SSIM和SD提升分别达10.26%、63.53%、22.21%、54.31%。源代码与数据集已公开于https://github.com/1hzf/MLS-UNet。
原文摘要 · Abstract (English)
Polarization image fusion combines S0 and DOLP images to reveal surface roughness and material properties through complementary texture features, which has important applications in camouflage recognition, tissue pathology analysis, surface defect detection and other fields. To intergrate coL-Splementary information from different polarized images in complex luminance environment, we propose a luminance-aware multi-scale network (MLSN). In the encoder stage, we propose a multi-scale spatial weight matrix through a brightness-branch , which dynamically weighted inject the luminance into the feature maps, solving the problem of inherent contrast difference in polarized images. The global-local feature fusion mechanism is designed at the bottleneck layer to perform windowed self-attention computation, to balance the global context and local details through residual linking in the feature dimension restructuring stage. In the decoder stage, to further improve the adaptability to complex lighting, we propose a Brightness-Enhancement module, establishing the mapping relationship between luminance distribution and texture features, realizing the nonlinear luminance correction of the fusion result. We also present MSP, an 1000 pairs of polarized images that covers 17 types of indoor and outdoor complex lighting scenes. MSP provides four-direction polarization raw maps, solving the scarcity of high-quality datasets in polarization image fusion. Extensive experiment on MSP, PIF and GAND datasets verify that the proposed MLSN outperms the state-of-the-art methods in subjective and objective evaluations, and the MS-SSIM and SD metircs are higher than the average values of other methods by 8.57%, 60.64%, 10.26%, 63.53%, 22.21%, and 54.31%, respectively. The source code and dataset is avalable at https://github.com/1hzf/MLS-UNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。