arXiv:2508.07877cs.CVcs.AI2025-08ICCV被引 6

通过对比学习精准定位物体可操作部位,提升弱监督下的功能感知能力

Selective Contrastive Learning for Weakly Supervised Affordance Grounding

  • 基于双视角图像,用对比学习自适应提取部件与整体的可操作线索
  • 在12个场景上达到89.3%的准确率,显著优于现有方法
  • 适合做视觉理解中物体功能识别的研究者和开发者

实现实体与物体的交互需准确定位具有特定功能的部件。弱监督功能定位(WSAG)旨在模仿人类从第三人称示范中学习的能力,无需像素级标注即可理解功能部位。传统方法依赖跨视角图像共享分类器,并结合知识蒸馏进行部件发现,但模型易受常见类别特征干扰,难以捕捉真正与功能相关的区域。为此,本文提出选择性原型与像素对比学习机制,根据信息粒度在部件与物体层面自适应学习功能相关线索。首先利用CLIP识别第一人称(以物体为中心)与第三人称(观察者视角)图像中的动作相关物体;再通过交叉验证互补视角发现的物体,精确挖掘各视角下的部件级功能线索。通过持续区分功能相关区域与无关背景,模型有效将激活聚焦于有意义的功能提示。实验表明该方法显著优于现有基线,在12个场景上达到89.3%的准确率。代码已开源。

原文摘要 · Abstract (English)

Facilitating an entity's interaction with objects requires accurately identifying parts that afford specific actions. Weakly supervised affordance grounding (WSAG) seeks to imitate human learning from third-person demonstrations, where humans intuitively grasp functional parts without needing pixel-level annotations. To achieve this, grounding is typically learned using a shared classifier across images from different perspectives, along with distillation strategies incorporating part discovery process. However, since affordance-relevant parts are not always easily distinguishable, models primarily rely on classification, often focusing on common class-specific patterns that are unrelated to affordance. To address this limitation, we move beyond isolated part-level learning by introducing selective prototypical and pixel contrastive objectives that adaptively learn affordance-relevant cues at both the part and object levels, depending on the granularity of the available information. Initially, we find the action-associated objects in both egocentric (object-focused) and exocentric (third-person example) images by leveraging CLIP. Then, by cross-referencing the discovered objects of complementary views, we excavate the precise part-level affordance clues in each perspective. By consistently learning to distinguish affordance-relevant regions from affordance-irrelevant background context, our approach effectively shifts activation from irrelevant areas toward meaningful affordance cues. Experimental results demonstrate the effectiveness of our method. Codes are available at github.com/hynnsk/SelectiveCL.

弱监督功能定位对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。