提出首个万级水下实例分割数据集与高效模型,解决水下图像分割难题。
Advancing Marine Research: UWSAM Framework and UIIS10K Dataset for Precise Underwater Instance Segmentation
- 用知识蒸馏将大模型能力迁移到小模型,提升水下视觉表征
- 在10个类别上达最优性能,比现有方法提升超过5个点
- 自动生成提示,无需人工标注,适合实际水下场景应用
随着大规模建模的发展,通用图像分割模型SAM在多种视觉任务中展现出巨大潜力。然而,由于缺乏水下领域专业知识,SAM及其变体在端到端水下实例分割任务中表现受限,且计算开销大,难以应用于水下环境。为此,我们构建了首个万级水下实例分割数据集UIIS10K,包含10,048张图像,涵盖10个类别,每张图均有像素级标注。同时提出UWSAM模型,通过基于掩码图注意力的水下知识蒸馏(MG-UKD)方法,将SAM ViT-Huge编码器的知识高效迁移到更小的ViT-Small编码器中,实现有效视觉表示学习。此外,设计端到端水下提示生成器(EUPG),可自动生成水下提示,无需人工提供前景点或框,从而精准定位水下目标并实现高效分割。大量实验证明,该模型在多个水下实例分割数据集上显著优于现有方法。代码与数据集已开源:https://github.com/LiamLian0727/UIIS10K。
原文摘要 · Abstract (English)
With recent breakthroughs in large-scale modeling, the Segment Anything Model (SAM) has demonstrated significant potential in a variety of visual applications. However, due to the lack of underwater domain expertise, SAM and its variants face performance limitations in end-to-end underwater instance segmentation tasks, while their higher computational requirements further hinder their application in underwater scenarios. To address this challenge, we propose a large-scale underwater instance segmentation dataset, UIIS10K, which includes 10,048 images with pixel-level annotations for 10 categories. Then, we introduce UWSAM, an efficient model designed for automatic and accurate segmentation of underwater instances. UWSAM efficiently distills knowledge from the SAM ViT-Huge image encoder into the smaller ViT-Small image encoder via the Mask GAT-based Underwater Knowledge Distillation (MG-UKD) method for effective visual representation learning. Furthermore, we design an End-to-end Underwater Prompt Generator (EUPG) for UWSAM, which automatically generates underwater prompts instead of explicitly providing foreground points or boxes as prompts, thus enabling the network to locate underwater instances accurately for efficient segmentation. Comprehensive experimental results show that our model is effective, achieving significant performance improvements over state-of-the-art methods on multiple underwater instance datasets. Datasets and codes are available at https://github.com/LiamLian0727/UIIS10K.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。