让机器人理解人类动作背后的因果逻辑,提升泛化能力。
CIVIL: Causal and Intuitive Visual Imitation Learning
- 人类用标记和语言说明行为原因,提供因果线索。
- 在模拟与真实环境中,性能优于现有基线模型。
- 适合需要高可靠性与适应性的机器人任务场景。
当前机器人通过模仿人类示范学习新任务,但仅观察行为而未理解其背后原因,导致因果混淆、学习效率低,且环境变化时表现失败。为此,我们提出新范式:不仅让人类示范动作,还引导其通过标记关键物体并用自然语言解释决策依据。基于此,提出的CIVIL算法利用增强演示数据过滤视觉信息,提取与人类一致的特征表示,并训练基于Transformer的策略网络。实验表明,该方法使机器人不仅能正确执行动作,还能识别因果因素,在仿真和真实世界中均优于现有基线。用户研究显示,新范式显著缩短学习时间,且在未见场景中表现更优。视频展示见项目主页:https://civil2025.github.io
原文摘要 · Abstract (English)
Today's robots attempt to learn new tasks by imitating human examples. These robots watch the human complete the task, and then try to match the actions taken by the human expert. However, this standard approach to visual imitation learning is fundamentally limited: the robot observes what the human does, but not why the human chooses those behaviors. Without understanding which features of the system or environment factor into the human's decisions, robot learners often misinterpret the human's examples. In practice, this results in causal confusion, inefficient learning, and robot policies that fail when the environment changes. We therefore propose a shift in perspective: instead of asking human teachers just to show what actions the robot should take, we also enable humans to intuitively indicate why they made those decisions. Under our paradigm human teachers attach markers to task-relevant objects and use natural language prompts to describe their state representation. Our proposed algorithm, CIVIL, leverages this augmented demonstration data to filter the robot's visual observations and extract a feature representation that aligns with the human teacher. CIVIL then applies these causal features to train a transformer-based policy that -- when tested on the robot -- is able to emulate human behaviors without being confused by visual distractors or irrelevant items. Our simulations and real-world experiments demonstrate that robots trained with CIVIL learn both what actions to take and why to take those actions, resulting in better performance than state-of-the-art baselines. From the human's perspective, our user study reveals that this new training paradigm actually reduces the total time required for the robot to learn the task, and also improves the robot's performance in previously unseen scenarios. See videos at our project website: https://civil2025.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。