机器人通过因果推理发现工具新用途,能跨任务灵活使用日常物品。
Creative Robot Tool Use by Counterfactual Reasoning

- 用物理模拟构建工具与任务的因果关系,识别关键特征。
- 通过反事实生成新物体,实现工具功能迁移和精准动作匹配。
- 适合需要创造性工具使用的机器人场景,如家务或救援。
我们提出一种用于创造性机器人工具使用的因果推理框架,能够识别适用于任务但超出其原始功能的工具。该框架首先在动力学模型中进行模拟实验,发现工具与任务间的因果关系。我们将因果发现分解为两个互补组件:基于视觉-语言模型的特征建议,以及通过针对性几何与物理特征扰动生成反事实工具。基于识别出的因果特征对新物体进行分类,并通过关键点匹配实现工具使用技能的迁移。通过在动力学模型中重构任务,我们的方法将工具使用建立在物理规律基础上。我们在多种场景中验证:用不同棍子够取远处物体、用各类物品舀取糖果、用盒子或箱子作为踏脚台取高处物品。基线对比表明,识别因果特征并将其锚定于物理属性,可提升工具选择可靠性与关键点迁移能力。
原文摘要 · Abstract (English)
We propose a causal reasoning framework for creative robot tool use where a suitable tool for a task is correctly identified for use beyond its primary objectives. The proposed framework first discovers the causal relationships between the tool and the task by conducting simulated experiments in a dynamics model. We decouple the causal discovery problem into two complementary components: VLM-based feature suggestion and counterfactual tool generation via targeted geometric and physical feature perturbations. Then, novel objects are classified based on identified causal features, and the tool use skill is transferred via keypoint matching conditioned on the identified causal features. By reconstructing the task in a dynamics model, our approach grounds tool use in the physics of the problem. We illustrate our approach in reaching a distant object with different sticks, scooping candies from a bowl using diverse items, and using different boxes or crates as stepping platforms to retrieve an object from a high shelf. Our baseline comparisons show that identifying causal features and grounding them in physical tool properties leads to more reliable tool selection and stronger skill keypoint transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。