用仿真-现实-仿真迭代训练，让机器人真实弹奏钢琴
Learning to Play Piano in the Real World
- 通过仿真-现实-仿真循环优化策略，提升真实世界泛化能力
- 在真实钢琴上成功演奏4首曲子，平均F1得分达0.881
- 为具身智能提供可复现的高精度操控基准，适合机器人学习研究者
为了实现机器人在真实世界中的类人操作能力，弹钢琴是一个极具挑战性的测试平台，要求策略性、精确性和流畅的动作。以往工作多在真实钢琴上使用手工设计控制器，或仅在模拟环境中评估学习方法。本文首次构建了一个基于学习方法并部署于真实灵巧机器人的钢琴演奏系统。采用Sim2Real2Sim方法，通过在仿真中训练策略、在真实世界部署、利用收集的真实数据更新仿真参数，实现迭代优化。实验表明，机器人能在真实钢琴上准确演奏《小星星》《生日快乐》《欢乐颂》《你是否已入睡》等曲目，平均F1得分为0.881。本工作为社区提供了实现类人操作的真实世界基准，鼓励更多研究关注此方向。代码与视频公开于www.lasr.org/research/learning-to-play-piano。
原文摘要 · Abstract (English)
Towards the grand challenge of achieving human-level manipulation in robots, playing piano is a compelling testbed that requires strategic, precise, and flowing movements. Over the years, several works demonstrated hand-designed controllers on real world piano playing, while other works evaluated robot learning approaches on simulated piano playing. In this work, we develop the first piano playing robotic system that makes use of learning approaches while also being deployed on a real world dexterous robot. Specifically, we use a Sim2Real2Sim approach where we iteratively alternate between training policies in simulation, deploying the policies in the real world, and use the collected real world data to update the parameters of the simulator. Using this approach we demonstrate that the robot can learn to play several piano pieces (including Are You Sleeping, Happy Birthday, Ode To Joy, and Twinkle Twinkle Little Star) in the real world accurately, reaching an average F1-score of 0.881. By providing this proof-of-concept, we want to encourage the community to adopt piano playing as a compelling benchmark towards human-level manipulation in the real world. We open-source our code and show additional videos at www.lasr.org/research/learning-to-play-piano .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。