用真实世界经验提升抓取策略,成功率提高37%
DexPIE: Stable Dexterous Policy Improvement from Real-World Experience

- 通过分阶段干预收集数据,覆盖更广探索空间
- 异步相对动作推理减少时间噪声,提升评估精度
- 基于连续最优指标微调,实现细粒度性能优化
灵巧操作因高维动作空间和复杂的接触动力学,给模仿学习带来挑战。仅依赖示范数据训练的策略在部署中易累积误差,且需大量专家数据才能可靠运行。为此,本文提出DexPIE,一种基于真实世界部署经验的灵巧策略后训练优化框架。首先,DexPIE通过适配灵巧手的干预系统与多阶段DAgger式数据采集,在初始与中间任务阶段实现有效探索覆盖,为策略评估提供可靠监督。为降低后训练轨迹与示范数据间的时间噪声,引入相对动作空间中的异步推理,使轨迹数据更贴近示范行为,帮助评判器学习更一致的值函数。最后,通过条件化连续最优性指标,使策略能以更精细的方式利用数据质量。在三个具挑战性的真实世界灵巧操作任务中,DexPIE相比基于示范的参考策略成功率提升37%,优于所有基线方法,展现出更强鲁棒性。代码与数据集将公开。
原文摘要 · Abstract (English)
Dexterous manipulation presents substantial challenges for imitation learning due to its high-dimensional action space and complex contact-rich dynamics. Policies trained purely from demonstrations often suffer from compounding errors during deployment and require large amounts of expert data to achieve reliable performance. To move beyond the limitations of demonstration data, in this work, we propose DexPIE, a post-training framework for dexterous policy improvement from experience collected through real-world deployment. First, DexPIE enables effective exploration coverage through a dexterous-hand-adapted intervention system and multi-stage DAgger-style data collection across initial and intermediate task stages, providing reliable supervision for accurate policy evaluation. To reduce temporal noise between post-training rollouts and demonstration data, we introduce asynchronous inference in the relative action space, which better aligns rollout data with demonstrated behavior and allows the critic to learn a value function induced by a more consistent underlying policy. Finally, DexPIE improves the policy through conditioning on a continuous optimality indicator, allowing the policy to leverage the quality of data in a more fine-grained manner. Across three challenging real-world dexterous manipulation tasks, DexPIE achieves a 37% improvement in success rate over the demonstration-based reference policy, outperforming all baseline methods and demonstrating stronger robustness. The source code and dataset will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。