用流匹配技术从低质量数据中学习机器人操作，实现复杂抓取任务。
Extremum Flow Matching for Offline Goal Conditioned Reinforcement Learning
- 基于流匹配的确定性迁移机制，可直接估计分布极值。
- 在OGBench和真实机器人上实现高维视觉下的连续动作规划，成功率超80%。
- 适合研究具身智能与低成本数据训练的团队使用。
模仿学习是赋予人形机器人通用能力的有前景方法，但其扩展受限于高质量专家示范数据的稀缺。该问题可通过利用易获取且多样性强的次优开放玩耍数据缓解。本文基于生成建模新进展——流匹配（Flow Matching），提出一种利用其确定性传输和任意源分布支持特性的方法，直接估计学习分布的最小或最大值。在此基础上，构建了多种基于流匹配的目标条件模仿与强化学习算法，策略同时依赖当前状态和目标观测。通过组合评论器、规划器、执行器或世界模型等核心组件，探索不同架构配置。在OGBench基准上评估，并分析数据收集过程中不同示范行为对2D非抓握推物任务性能的影响。进一步在真实硬件上验证：将方法部署于Talos人形机器人，在真实厨房环境中完成包含一系列拾取放置与柔性物体操作的复杂任务，仅依赖高维图像观测。实验视频与代码见：https://hucebot.github.io/extremum_flow_matching_website/
原文摘要 · Abstract (English)
Imitation learning is a promising approach for enabling generalist capabilities in humanoid robots, but its scaling is fundamentally constrained by the scarcity of high-quality expert demonstrations. This limitation can be mitigated by leveraging suboptimal, open-ended play data, often easier to collect and offering greater diversity. This work builds upon recent advances in generative modeling, specifically Flow Matching, an alternative to Diffusion models. We introduce a method for estimating the minimum or maximum of the learned distribution by leveraging the unique properties of Flow Matching, namely, deterministic transport and support for arbitrary source distributions. We apply this method to develop several goal-conditioned imitation and reinforcement learning algorithms based on Flow Matching, where policies are conditioned on both current and goal observations. We explore and compare different architectural configurations by combining core components, such as critic, planner, actor, or world model, in various ways. We evaluated our agents on the OGBench benchmark and analyzed how different demonstration behaviors during data collection affect performance in a 2D non-prehensile pushing task. Furthermore, we validated our approach on real hardware by deploying it on the Talos humanoid robot to perform complex manipulation tasks based on high-dimensional image observations, featuring a sequence of pick-and-place and articulated object manipulation in a realistic kitchen environment. Experimental videos and code are available at: https://hucebot.github.io/extremum_flow_matching_website/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。