arXiv:2503.23786cs.CVcs.AI2025-03被引 4

提升高分辨率图像的无类别分割细节,无需手动提示。

MGD-SAM2: Multi-view Guided Detail-enhanced Segment Anything Model 2 for High-Resolution Class-agnostic Segmentation

  • 引入多视角特征交互,融合全局与局部信息增强细节感知。
  • 在多个高分辨率数据集上实现更精确的分割,优于原始SAM2。
  • 适合需要高精度无类别分割的应用,如遥感、医学影像分析。

作为视觉基础模型,分割一切模型(SAMs)在多种图像分析任务中表现出色。然而,由于直接处理高分辨率输入的能力受限及掩码预测分辨率低,且依赖准确的人工提示,现有SAMs在高分辨率无类别分割(HRCS)任务中仍面临细粒度细节分割难题。为此,我们提出MGD-SAM2,通过将SAM2与全局图像和局部块之间的多视角特征交互相结合,实现精准分割。MGD-SAM2在预训练SAM2基础上引入四个新模块:多视角感知适配器(MPAdapter)、多视角互补增强模块(MCEM)、分层多视角交互模块(HMIM)和细节修复模块(DRM)。首先,MPAdapter增强对高分辨率图像中局部细节与全局语义的提取能力;随后,MCEM与HMIM通过跨尺度多视角特征聚合,进一步挖掘局部纹理与全局上下文;最后,DRM逐步恢复高分辨率掩码预测,弥补直接上采样导致的细粒度信息丢失。实验表明,该模型在多个高分辨率与常规分辨率数据集上均表现优异,具备更强泛化能力。代码将发布于https://github.com/sevenshr/MGD-SAM2。

原文摘要 · Abstract (English)

Segment Anything Models (SAMs), as vision foundation models, have demonstrated remarkable performance across various image analysis tasks. Despite their strong generalization capabilities, SAMs encounter challenges in fine-grained detail segmentation for high-resolution class-independent segmentation (HRCS), due to the limitations in the direct processing of high-resolution inputs and low-resolution mask predictions, and the reliance on accurate manual prompts. To address these limitations, we propose MGD-SAM2 which integrates SAM2 with multi-view feature interaction between a global image and local patches to achieve precise segmentation. MGD-SAM2 incorporates the pre-trained SAM2 with four novel modules: the Multi-view Perception Adapter (MPAdapter), the Multi-view Complementary Enhancement Module (MCEM), the Hierarchical Multi-view Interaction Module (HMIM), and the Detail Refinement Module (DRM). Specifically, we first introduce MPAdapter to adapt the SAM2 encoder for enhanced extraction of local details and global semantics in HRCS images. Then, MCEM and HMIM are proposed to further exploit local texture and global context by aggregating multi-view features within and across multi-scales. Finally, DRM is designed to generate gradually restored high-resolution mask predictions, compensating for the loss of fine-grained details resulting from directly upsampling the low-resolution prediction maps. Experimental results demonstrate the superior performance and strong generalization of our model on multiple high-resolution and normal-resolution datasets. Code will be available at https://github.com/sevenshr/MGD-SAM2.

分割模型高分辨率多视角细节增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。