用30分钟真实数据让机器人快速学会精准钢琴演奏
HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies
- 先用仿真训练基础动作，再通过物理试错修正手指位置
- 实测5首曲子均成功弹奏，比直接用仿真策略提升1.8倍
- 适合想快速部署高精度机械手的科研与工业应用
多年来，多指灵巧操作一直是机器人领域的重大挑战。尽管强化学习和仿真到现实的迁移技术前景广阔，但面对毫米级精度要求的任务（如双手钢琴演奏），已有策略仍常失败。本文提出HandelBot框架，采用两阶段快速适应：首先基于物理试运行，通过调整横向指关节修正空间对齐；接着利用残差强化学习自主学习微调动作。在五首公认曲目的硬件实验中，系统成功实现精准双手演奏。相比直接使用仿真策略，性能提升1.8倍，仅需30分钟真实交互数据即可完成适配。
原文摘要 · Abstract (English)
Mastering dexterous manipulation with multi-fingered hands has been a grand challenge in robotics for decades. Despite its potential, the difficulty of collecting high-quality data remains a primary bottleneck for high-precision tasks. While reinforcement learning and simulation-to-real-world transfer offer a promising alternative, the transferred policies often fail for tasks demanding millimeter-scale precision, such as bimanual piano playing. In this work, we introduce HandelBot, a framework that combines a simulation policy and rapid adaptation through a two-stage pipeline. Starting from a simulation-trained policy, we first apply a structured refinement stage to correct spatial alignments by adjusting lateral finger joints based on physical rollouts. Next, we use residual reinforcement learning to autonomously learn fine-grained corrective actions. Through extensive hardware experiments across five recognized songs, we demonstrate that HandelBot can successfully perform precise bimanual piano playing. Our system outperforms direct simulation deployment by a factor of 1.8x and requires only 30 minutes of physical interaction data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。