用人体姿态引导，让分割模型在人群遮挡下更准更稳。
SAM-pose2seg: Pose-Guided Human Instance Segmentation in Crowds
- 用可见关键点引导迭代修正，改进原版SAM的遮挡适应性。
- 仅需1个可见关键点即可准确分割，多数据集上精度提升显著。
- 轻量微调适配，保留原始模型泛化能力，适合实际场景部署。
Segment Anything (SAM) 为人体分割提供了前所未有的基础，但在遮挡情况下表现不佳，关键点可能部分或完全不可见。我们对 SAM 2.1 进行最小修改，实现姿态引导的分割，保持其强大泛化能力。通过名为 PoseMaskRefine 的微调策略，将高可见性姿态关键点融入 SAM 原有的迭代修正过程，显著提升多个数据集上的鲁棒性和准确性。推理时，仅选取三个最高可见性的关键点作为提示，降低对常见错误（如缺失肢体或衣物误分类）的敏感度，并可实现仅凭单个关键点生成准确掩码。结果表明，姿态引导的 SAM 微调能有效实现遮挡感知的人体实例分割，同时保留原始模型的泛化性能。代码与预训练模型将在 https://mirapurkrabek.github.io/BBox-Mask-Pose/ 公开。
原文摘要 · Abstract (English)
Segment Anything (SAM) provides an unprecedented foundation for human segmentation, but may struggle under occlusion, where keypoints may be partially or fully invisible. We adapt SAM 2.1 for pose-guided segmentation with minimal encoder modifications, retaining its strong generalization. Using a fine-tuning strategy called PoseMaskRefine, we incorporate pose keypoints with high visibility into the iterative correction process originally employed by SAM, yielding improved robustness and accuracy across multiple datasets. During inference, we simplify prompting by selecting only the three keypoints with the highest visibility. This strategy reduces sensitivity to common errors, such as missing body parts or misclassified clothing, and allows accurate mask prediction from as few as a single keypoint. Our results demonstrate that pose-guided fine-tuning of SAM enables effective, occlusion-aware human segmentation while preserving the generalization capabilities of the original model. The code and pretrained models will be available at https://mirapurkrabek.github.io/BBox-Mask-Pose/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。