通过技能信息提升机器人操作的泛化能力
Skill-Aware Diffusion for Generalizable Robotic Manipulation
- 用可学习的技能标记显式建模技能级特征
- 在仿真和真实环境中均实现良好泛化性能
- 适合需要跨任务适应的机器人控制研究
鲁棒的泛化能力对机器人适应多样化环境至关重要。现有方法通常通过扩大数据量和网络规模来提升泛化性，但独立处理各项任务，忽略了技能层级信息。我们观察到同一技能内的任务具有相似运动模式，提出技能感知扩散模型（SADiff），显式引入技能级信息以增强泛化能力。SADiff通过带有可学习技能标记的技能感知编码模块学习技能特定表示，并利用技能约束的扩散模型生成以物体为中心的运动流。此外，引入技能检索转换策略，利用技能特定轨迹先验优化从2D运动流到可执行3D动作的映射。为进一步评估与真实场景迁移，我们构建了IsaacSkill数据集，包含基础机器人技能的高保真数据。仿真与真实世界实验表明，SADiff在多种操作任务中表现优异且具备强泛化能力。代码、数据与视频见https://sites.google.com/view/sa-diff。
原文摘要 · Abstract (English)
Robust generalization in robotic manipulation is crucial for robots to adapt flexibly to diverse environments. Existing methods usually improve generalization by scaling data and networks, but model tasks independently and overlook skill-level information. Observing that tasks within the same skill share similar motion patterns, we propose Skill-Aware Diffusion (SADiff), which explicitly incorporates skill-level information to improve generalization. SADiff learns skill-specific representations through a skill-aware encoding module with learnable skill tokens, and conditions a skill-constrained diffusion model to generate object-centric motion flow. A skill-retrieval transformation strategy further exploits skill-specific trajectory priors to refine the mapping from 2D motion flow to executable 3D actions. Furthermore, we introduce IsaacSkill, a high-fidelity dataset containing fundamental robotic skills for comprehensive evaluation and sim-to-real transfer. Experiments in simulation and real-world settings show that SADiff achieves good performance and generalization across various manipulation tasks. Code, data, and videos are available at https://sites.google.com/view/sa-diff.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。