用频域提示提升SAM,高效分割复杂海域动物。
HFP-SAM: Hierarchical Frequency Prompted SAM for Efficient Marine Animal Segmentation
- 通过频域先验掩码向冻结的SAM注入海洋场景信息。
- 频域分析生成关键区域点提示,显著提升细粒度分割精度。
- 适合需要高精度海洋生物分割的研究与应用者。
海洋动物分割(MAS)旨在从复杂的海洋环境中识别并分割出海洋生物。以往基于深度学习的方法普遍存在长距离建模困难的问题。最近,通用图像分割模型Segment Anything Model(SAM)受到广泛关注,但其在细粒度细节和频率信息感知方面表现不足。为此,本文提出一种新型学习框架——分层频域提示SAM(HFP-SAM),用于高性能海洋动物分割。首先,设计频域引导适配器(FGA),通过频域先验掩码高效地将海洋场景信息注入冻结的SAM主干网络。其次,引入频域感知点选择(FPS),基于频域分析生成显著区域,并与SAM的粗分割结果结合生成点提示,输入到SAM解码器中实现精细预测。最后,为获得全面的分割掩码,提出全视图马尔可夫模块(FVM),以线性计算复杂度高效提取空间与通道上下文信息。在四个公开数据集上的大量实验验证了该方法的优越性能。源代码已开源:https://github.com/Drchip61/TIP-HFP-SAM。
原文摘要 · Abstract (English)
Marine Animal Segmentation (MAS) aims at identifying and segmenting marine animals from complex marine environments. Most of previous deep learning-based MAS methods struggle with the long-distance modeling issue. Recently, Segment Anything Model (SAM) has gained popularity in general image segmentation. However, it lacks of perceiving fine-grained details and frequency information. To this end, we propose a novel learning framework, named Hierarchical Frequency Prompted SAM (HFP-SAM) for high-performance MAS. First, we design a Frequency Guided Adapter (FGA) to efficiently inject marine scene information into the frozen SAM backbone through frequency domain prior masks. Additionally, we introduce a Frequency-aware Point Selection (FPS) to generate highlighted regions through frequency analysis. These regions are combined with the coarse predictions of SAM to generate point prompts and integrate into SAM's decoder for fine predictions. Finally, to obtain comprehensive segmentation masks, we introduce a Full-View Mamba (FVM) to efficiently extract spatial and channel contextual information with linear computational complexity. Extensive experiments on four public datasets demonstrate the superior performance of our approach. The source code is publicly available at https://github.com/Drchip61/TIP-HFP-SAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。