用生成模型补足红外图像信息,提升少样本分割效果。
Generative Model-Based Fusion for Improved Few-Shot Semantic Segmentation of Infrared Images
- 通过生成模型合成多通道数据,增强红外图像对比度。
- 在多个红外数据集上优于当前最优方法,显著提升分割精度。
- 适合处理数据稀缺、无配对可见光图像的红外场景。
红外(IR)成像广泛应用于自动驾驶、消防安全和国防等领域,其语义分割备受关注。然而,该任务面临数据稀疏、与自然图像对比度差异大、通道数不同,以及特定场景中出现数据库未涵盖新类等挑战。少样本分割(FSS)可通过少量标注的支持样本实现查询图像分割,但现有方法依赖配对的可见光RGB图像,而此类数据在部分应用中难以获取。本文提出基于生成建模与融合技术的新策略:首先生成辅助数据以补充红外图像的通道信息,提升对比度;其次通过红外数据生成实现数据增强。此外,设计新型融合集成模块,进一步优化支持与查询图像间的关联学习。实验在多个红外数据集上验证,性能超越现有SOTA方法。
原文摘要 · Abstract (English)
Infrared (IR) imaging is commonly used in various scenarios, including autonomous driving, fire safety and defense applications. Thus, semantic segmentation of such images is of great interest. However, this task faces several challenges, including data scarcity, differing contrast and input channel number compared to natural images, and emergence of classes not represented in databases in certain scenarios, such as defense applications. Few-shot segmentation (FSS) provides a framework to overcome these issues by segmenting query images using a few labeled support samples. However, existing FSS models for IR images require paired visible RGB images, which is a major limitation since acquiring such paired data is difficult or impossible in some applications. In this work, we develop new strategies for FSS of IR images by using generative modeling and fusion techniques. To this end, we propose to synthesize auxiliary data to provide additional channel information to complement the limited contrast in the IR images, as well as IR data synthesis for data augmentation. Here, the former helps the FSS model to better capture the relationship between the support and query sets, while the latter addresses the issue of data scarcity. Finally, to further improve the former aspect, we propose a novel fusion ensemble module for integrating the two different modalities. Our methods are evaluated on different IR datasets, and improve upon the state-of-the-art (SOTA) FSS models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。