通过跨图像动作对齐，让机器人零样本识别物体操作性。
RAAP: Retrieval-Augmented Affordance Prediction with Cross-Image Action Alignment
- 分离接触点与动作方向，用密集对应传递接触信息
- 仅需每任务数十样本，零样本下仍保持稳定性能
- 适合需要快速泛化到新物体的机器人交互场景
理解物体操作性对机器人在多样且非结构化环境中执行有目的、精细化交互至关重要。现有方法或依赖检索（易受稀疏性和覆盖缺口影响），或依赖大规模模型（常误定位接触点、错判接触后动作），导致泛化能力受限。我们提出检索增强型操作性预测框架RAAP，将操作性检索与基于对齐的学习统一起来。通过解耦静态接触定位与动态动作方向，RAAP利用密集对应传递接触点，并通过检索增强的对齐模型，以双重加权注意力融合多个参考样本预测动作方向。在仅使用DROID和HOI4D的精简子集（每任务仅需数十样本）训练下，RAAP在未见物体和类别上表现一致，实现在仿真与真实世界中的零样本机器人操作。项目主页：https://github.com/SEU-VIPGroup/RAAP。
原文摘要 · Abstract (English)
Understanding object affordances is essential for enabling robots to perform purposeful and fine-grained interactions in diverse and unstructured environments. However, existing approaches either rely on retrieval, which is fragile due to sparsity and coverage gaps, or on large-scale models, which frequently mislocalize contact points and mispredict post-contact actions when applied to unseen categories, thereby hindering robust generalization. We introduce Retrieval-Augmented Affordance Prediction (RAAP), a framework that unifies affordance retrieval with alignment-based learning. By decoupling static contact localization and dynamic action direction, RAAP transfers contact points via dense correspondence and predicts action directions through a retrieval-augmented alignment model that consolidates multiple references with dual-weighted attention. Trained on compact subsets of DROID and HOI4D with as few as tens of samples per task, RAAP achieves consistent performance across unseen objects and categories, and enables zero-shot robotic manipulation in both simulation and the real world. Project website: https://github.com/SEU-VIPGroup/RAAP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。