通过模态级探索提升机器人自我改进能力
SIME: Enhancing Policy Self-Improvement with Modal-level Exploration
- 在策略执行中引入模态级探索机制，生成更多样化的交互数据
- 选择高质量、高价值的试错片段进行学习，显著提升改进效率
- 适用于需持续优化的机器人控制场景，降低训练成本
机器人自我改进依赖于初始的人类示范数据，并通过与环境的交互逐步提升能力，类似于人类通过反复练习提升技能。然而，有效自我改进面临挑战，主要源于机器人在交互中倾向于重复已有行为，难以生成新且有价值的数据用于学习。本文提出，成功自我改进的关键在于模态级探索与数据选择。通过在策略执行中引入模态级探索机制，机器人可产生更丰富、多模态的交互。同时，从这些交互中筛选出最具价值的试错样本和高质量片段用于学习。我们在仿真基准和真实世界实验中均验证了该方法的有效性。该能力将使我们以更低成本开发出更鲁棒、成功率更高的机器人控制策略。代码与实验脚本已公开于 https://ericjin2002.github.io/SIME/
原文摘要 · Abstract (English)
Self-improvement requires robotic systems to initially learn from human-provided data and then gradually enhance their capabilities through interaction with the environment. This is similar to how humans improve their skills through continuous practice. However, achieving effective self-improvement is challenging, primarily because robots tend to repeat their existing abilities during interactions, often failing to generate new, valuable data for learning. In this paper, we identify the key to successful self-improvement: modal-level exploration and data selection. By incorporating a modal-level exploration mechanism during policy execution, the robot can produce more diverse and multi-modal interactions. At the same time, we select the most valuable trials and high-quality segments from these interactions for learning. We successfully demonstrate effective robot self-improvement on both simulation benchmarks and real-world experiments. The capability for self-improvement will enable us to develop more robust and high-success-rate robotic control strategies at a lower cost. Our code and experiment scripts are available at https://ericjin2002.github.io/SIME/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。