统一模型实现高精度图像分割与抠图,仅用少量新增参数。
Segment and Matte Anything in a Unified Model

- 设计多视角定位编码器与适配器,精细恢复边界细节。
- 在多个基准上达到顶尖性能,分割与抠图同步优化。
- 适合需要高精度交互式图像处理的开发者与研究者。
Segment Anything (SAM) 在训练超过十亿个掩码后,展现出零样本泛化与灵活提示的能力,但其掩码预测精度在真实应用中仍不足。尽管已有多种精修模块提升性能,但在单一统一框架内实现高精度物体轮廓提取仍是未解难题。此外,基于用户提示生成细粒度透明度掩码的交互式图像抠图,尚未在 SAM 背景下被探索。近期研究表明分割与抠图存在强相关性,为统一建模提供了可行性。本文提出 Segment And Matte Anything (SAMA),作为 SAM 的轻量级扩展,以极小额外参数实现高质量交互式分割与抠图。其多视角定位编码器(MVLE)捕捉局部细节特征,定位适配器(Local-Adapter)通过恢复细微边界提升掩码质量,并在架构中集成双预测头,同步生成分割与抠图掩码。在整合公开数据源的多样化数据集上训练,SAMA 在多个分割与抠图基准上表现领先,展现其在多种下游任务中的适应性与有效性。
原文摘要 · Abstract (English)
Segment Anything (SAM) has recently pushed the boundaries of segmentation by demonstrating zero-shot generalization and flexible prompting after training on over one billion masks. Despite this, its mask prediction accuracy often falls short of the precision required in real-world applications. While several refinement modules have been proposed to boost SAM's segmentation quality, achieving highly accurate object delineation within a single, unified framework remains an open challenge. Furthermore, interactive image matting, which aims to generate fine-grained alpha mattes guided by diverse user hints, has not yet been explored in the context of SAM. Insights from recent studies highlight strong correlations between segmentation and matting, suggesting the feasibility of a unified model capable of both tasks. In this paper, we introduce Segment And Matte Anything (SAMA), a lightweight extension of SAM that delivers high-quality interactive image segmentation and matting with minimal extra parameters. Our Multi-View Localization Encoder (MVLE) captures detailed features from local views, while the Localization Adapter (Local-Adapter) refines mask outputs by recovering subtle boundary details. We also incorporate two prediction heads for each task into the architecture to generate segmentation and matting masks, simultaneously. Trained on a diverse dataset aggregated from publicly available sources, SAMA achieves state-of-the-art performance across multiple segmentation and matting benchmarks, showcasing its adaptability and effectiveness in a wide range of downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。