用外部知识提升机器人模仿学习泛化能力，少样本也能适应新环境。
Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions
- 引入语义关键点图作为知识模板，抽象同类物体共性特征。
- 仅需四分之一专家数据，性能超过基于图像的扩散策略。
- 适合在真实场景中追求高效、鲁棒机器人学习的研究者。
模仿学习在机器人操作中表现强大，但受限于有限专家示范中的物体特异性。为此，我们提出知识驱动的模仿学习框架，利用外部结构语义知识抽象同一类别物体的表征。提出新型语义关键点图作为知识模板，并开发粗到细的模板匹配算法，同时优化结构一致性和语义相似性。在三个真实机器人操作任务上评估，该方法性能更优，仅需图像基扩散策略四分之一的专家示范即达其效果。大量实验进一步验证其在新物体、背景与光照条件下的鲁棒性。本工作开创了真实场景下数据高效机器人学习的知识驱动路径。代码与更多资料见 https://knowledge-driven.github.io/。
原文摘要 · Abstract (English)
Imitation learning has emerged as a powerful paradigm in robot manipulation, yet its generalization capability remains constrained by object-specific dependencies in limited expert demonstrations. To address this challenge, we propose knowledge-driven imitation learning, a framework that leverages external structural semantic knowledge to abstract object representations within the same category. We introduce a novel semantic keypoint graph as a knowledge template and develop a coarse-to-fine template-matching algorithm that optimizes both structural consistency and semantic similarity. Evaluated on three real-world robotic manipulation tasks, our method achieves superior performance, surpassing image-based diffusion policies with only one-quarter of the expert demonstrations. Extensive experiments further demonstrate its robustness across novel objects, backgrounds, and lighting conditions. This work pioneers a knowledge-driven approach to data-efficient robotic learning in real-world settings. Code and more materials are available on https://knowledge-driven.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。