让模型识别被遮挡物体的完整轮廓，只需换前端检测器即可
Segment Anything, Even Occluded
- 用SAM当通用掩码解码器，兼容不同检测器
- 在30万张合成图上训练，零样本性能超预期
- 适合做遮挡场景下的目标识别与机器人操作
无模态实例分割旨在检测并分割图像中物体的可见与不可见部分，在自动驾驶、机器人操作和场景理解中至关重要。现有方法需联合训练前端检测器与掩码解码器，灵活性差且难以利用预训练检测器的优势。为此，我们提出SAMEO框架，将Segment Anything Model（SAM）作为通用掩码解码器，可对接多种前端检测器，实现对部分遮挡物体的完整掩码预测。针对无模态分割数据集有限的问题，我们构建了Amodal-LVIS，一个基于模态LVIS和LVVIS数据集生成的大型合成数据集，包含30万张图像。实验表明，基于该扩展数据集训练后，该方法在COCOA-cls和D2SA两个基准上均取得优异的零样本性能，展现出对未见场景的强大泛化能力。
原文摘要 · Abstract (English)
Amodal instance segmentation, which aims to detect and segment both visible and invisible parts of objects in images, plays a crucial role in various applications including autonomous driving, robotic manipulation, and scene understanding. While existing methods require training both front-end detectors and mask decoders jointly, this approach lacks flexibility and fails to leverage the strengths of pre-existing modal detectors. To address this limitation, we propose SAMEO, a novel framework that adapts the Segment Anything Model (SAM) as a versatile mask decoder capable of interfacing with various front-end detectors to enable mask prediction even for partially occluded objects. Acknowledging the constraints of limited amodal segmentation datasets, we introduce Amodal-LVIS, a large-scale synthetic dataset comprising 300K images derived from the modal LVIS and LVVIS datasets. This dataset significantly expands the training data available for amodal segmentation research. Our experimental results demonstrate that our approach, when trained on the newly extended dataset, including Amodal-LVIS, achieves remarkable zero-shot performance on both COCOA-cls and D2SA benchmarks, highlighting its potential for generalization to unseen scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。