针对水下图像少样本分割,提出增强特征的网络框架。
FSSUWNet: Mitigating the Fragility of Pre-trained Models with Feature Enhancement for Few-Shot Semantic Segmentation in Underwater Images
- 设计双编码器结构,融合主模型与增强特征提取器。
- 在1-shot和5-shot场景下,mIoU分别提升2.8%和2.6%。
- 适合水下视觉、少样本学习等领域的研究者参考。
少样本语义分割(FSS)旨在仅用少量标注样本对新类别进行分割,已在数据稀缺领域取得进展。然而,本文指出现有FSS方法在水下环境中泛化能力差,原因在于预训练模型提取的先验特征因水下图像的独特挑战而脆弱。为此,我们提出专为水下图像设计的FSSUWNet框架,通过特征增强来缓解此问题。该框架整合互补特征,强调低层与高层图像特性。除使用预训练模型作为主编码器外,还引入辅助编码器——特征增强编码器,以提取更适配水下场景的补充特征。此外,设计简单有效的特征对齐模块,提供全局先验知识并实现高低层特征在维度上的对齐。鉴于水下图像数据稀缺,我们基于水下图像分割数据集构建了交叉验证版本。在公开水下分割数据集上的大量实验表明,本方法达到当前最优性能。例如,在1-shot和5-shot场景下,本方法相比之前最佳方法在均交并比(mIoU)上分别提升2.8%和2.6%。代码已开源:https://github.com/lizhh268/FSSUWNet。
原文摘要 · Abstract (English)
Few-Shot Semantic Segmentation (FSS), which focuses on segmenting new classes in images using only a limited number of annotated examples, has recently progressed in data-scarce domains. However, in this work, we show that the existing FSS methods often struggle to generalize to underwater environments. Specifically, the prior features extracted by pre-trained models used as feature extractors are fragile due to the unique challenges of underwater images. To address this, we propose FSSUWNet, a tailored FSS framework for underwater images with feature enhancement. FSSUWNet exploits the integration of complementary features, emphasizing both low-level and high-level image characteristics. In addition to employing a pre-trained model as the primary encoder, we propose an auxiliary encoder called Feature Enhanced Encoder which extracts complementary features to better adapt to underwater scene characteristics. Furthermore, a simple and effective Feature Alignment Module aims to provide global prior knowledge and align low-level features with high-level features in dimensions. Given the scarcity of underwater images, we introduce a cross-validation dataset version based on the Segmentation of Underwater Imagery dataset. Extensive experiments on public underwater segmentation datasets demonstrate that our approach achieves state-of-the-art performance. For example, our method outperforms the previous best method by 2.8% and 2.6% in terms of the mean Intersection over Union metric for 1-shot and 5-shot scenarios in the datasets, respectively. Our implementation is available at https://github.com/lizhh268/FSSUWNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。