让机器人通过物体功能而非外观来理解世界,提升新场景适应能力。
What Objects Enable, Not What They Are: Functional Latent Spaces for Affordance Reasoning

- 将视觉信息映射到以功能为组织的潜在空间,按可移动性等属性分类
- 对已有功能推理准确率达94%,新功能识别准确率从70%提升至90%以上
- 支持未知场景自动发现新功能,推理速度提升100倍,适合机器人规划应用
现有机器人规划系统依赖外观感知,将视觉观察编码到以物体外观为中心的潜在空间(如根据外形识别‘购物车’)。但规划需关注任务相关的物体功能(如是否可移动),而外观型潜在空间无法捕捉这些信息,导致难以泛化到新交互场景。本文提出A4D,将视觉观测映射到以功能(如‘可移动’)为结构的共享潜在空间。通过测量观测与功能之间的距离,A4D推断物体的任务相关功能。同时引入功能发现机制,当现有功能不足时自动扩展潜在空间。A4D利用功能空间中的距离量化推断不确定性,并选择性触发功能发现。在多个涉及多样化且未见功能的规划任务中评估,A4D在已有功能上的推理准确率达94%,优于当前最优方法超15个百分点;新功能识别准确率从70%提升至90%以上,仅用不到10%的原始训练数据;推理速度提升100倍。代码、视频及数据见:https://A4Dance-reasoning.github.io。
原文摘要 · Abstract (English)
Existing robot planning systems rely on appearance-based reasoning, where visual observations are encoded into latent spaces organized around object appearances (e.g., recognizing a "cart" based on how it looks). However, planning requires reasoning about task-relevant functionalities of objects (e.g., whether an object is "movable"), which appearance-based latent spaces do not capture. As a result, existing approaches struggle to generalize to novel robot-object interactions. We address this limited generalizability through affordance reasoning, enabling planning based on task-relevant object functionalities instead of appearance alone. We introduce A4D, which maps visual observations into a shared latent space structured around affordances (e.g., "movable"). By projecting visual observations into this functional latent space and measuring their proximity to affordances, A4D infers functionalities relevant to the observed object. Furthermore, we introduce an affordance discovery mechanism that expands the latent space to handle unseen scenarios where existing affordances are insufficient. A4D uses proximity in the functional latent space to quantify uncertainty in affordance inference and selectively triggers affordance discovery. We evaluate A4D across several planning tasks involving diverse and unseen affordances. A4D achieves 94% inference accuracy on existing affordances outperforming state-of-the-art approaches by over 15% points, improves new-affordance inference accuracy from 70% to over 90% with fewer than 10% of the original training data, and enables 100x faster inference. Code, videos, and data available at: https://A4Dance-reasoning.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。