用SAM扩展到动作可及性识别，提升泛化能力
AffordanceSAM: Segment Anything Once More in Affordance Grounding
- 基于SAM设计适配模块，三阶段训练迁移分割能力
- 在AGD20K上达当前最优，显著提升新场景泛化性能
- 适合需要高泛化动作理解的机器人视觉应用
构建通用的动作可及性接地模型以识别物体上的可操作区域，对真实世界应用至关重要。现有方法分为弱监督和全监督两类：前者需复杂框架设计且无法推理新动作；后者虽结构简单，但受限于标注数据少，且组件多从头训练。本研究聚焦全监督动作可及性接地，提出AffordanceSAM，将SAM的分割泛化能力拓展至动作可及性任务。具体地，设计了动作适配模块，并构建了一个粗到细标注的数据集C2F-Aff，通过三阶段训练实现性能迁移。实验表明，AffordanceSAM在AGD20K基准上达到当前最优（SOTA）表现，具备强泛化能力。
原文摘要 · Abstract (English)
Building a generalized affordance grounding model to identify actionable regions on objects is vital for real-world applications. Existing methods to train the model can be divided into weakly and fully supervised ways. However, the former method requires a complex training framework design and can not infer new actions without an auxiliary prior. While the latter often struggle with limited annotated data and components trained from scratch despite being simpler. This study focuses on fully supervised affordance grounding and overcomes its limitations by proposing AffordanceSAM, which extends SAM's generalization capacity in segmentation to affordance grounding. Specifically, we design an affordance-adaption module and curate a coarse-to-fine annotated dataset called C2F-Aff to thoroughly transfer SAM's robust performance to affordance in a three-stage training manner. Experimental results confirm that AffordanceSAM achieves state-of-the-art (SOTA) performance on the AGD20K benchmark and exhibits strong generalized capacity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。