让机器人通过预见动作后果，用一套策略完成100多个任务
FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation
- 用多模态目标条件+动作预判增强策略
- 仿真与真实场景中成功率提升最高达41%
- 适合需要少样本泛化和高可靠性操作的研究者
多任务模仿学习在机器人操作中展现出巨大潜力，使单一策略能执行多种任务，简化部署并提升适应性。但关键挑战仍存，如保持动作可靠性（避免偏离正常轨迹的异常序列）及在少量专家示范下泛化到未见任务。为此，我们提出前瞻增强型操作策略（FoAM），首次将多模态目标条件作为输入，并引入前瞻增强机制，不仅重建动作，还让智能体推理动作的视觉后果（状态），学习更具表现力的嵌入表示以捕捉细微任务差异。在超过100个仿真与真实世界任务中进行的大量实验表明，FoAM显著提升多任务模仿学习性能，成功率相比最先进基线最高提升41%。同时，我们发布了包含10个场景、80多个挑战性任务的仿真环境，用于操作策略训练与评估。项目详情见projFoAM.github.io。
原文摘要 · Abstract (English)
Multi-task imitation learning (MTIL) has shown significant potential in robotic manipulation by enabling agents to perform various tasks using a single policy. This simplifies the policy deployment and enhances the agent's adaptability across different scenarios. However, key challenges remain, such as maintaining action reliability (e.g., avoiding abnormal action sequences that deviate from nominal task trajectories) and generalizing to unseen tasks with a few expert demonstrations. To address these challenges, we introduce the Foresight-Augmented Manipulation Policy (FoAM), a novel MTIL policy that pioneers the use of multi-modal goal condition as input and introduces a foresight augmentation in addition to the general action reconstruction. FoAM enables the agent to reason about the visual consequences (states) of its actions and learn more expressive embedding that captures nuanced task variations. Extensive experiments on over 100 tasks in simulation and real-world settings demonstrate that FoAM significantly enhances MTIL policy performance, outperforming state-of-the-art baselines by up to 41% in success rate. Meanwhile, we released our simulation suites, including a total of 10 scenarios and over 80 challenging tasks designed for manipulation policy training and evaluation. See the project homepage projFoAM.github.io for project details.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。