实测参数比扩大随机范围更有效,小样本测量即可显著提升机器人迁移性能。
How Should a Simulation-to-Reality Transfer Budget Be Spent?

- 用少量真实数据识别系统参数,优于广泛随机化训练。
- 少量测量便能关闭大部分仿真到现实的性能差距。
- 适合追求高效迁移的机器人学习研究者参考。
仿真到现实的迁移(sim-to-real transfer)是机器人学习的核心挑战。本文研究了在系统辨识与领域随机化之间分配真实机器人测量时间的权衡。在可控的仿真摆杆环境中,隐藏参数模拟真实机器人,实验对比了辨识滚动次数与随机化分布宽度的影响。在不同现实差距和噪声水平下,测量预算起决定性作用:少量辨识滚动即可消除大部分迁移差距;一旦有真实数据,基于估计参数训练的策略表现最优,即使随机化范围包含真实参数也无法替代测量。该结论适用于动力学可辨识且仅两个参数未知的良性场景,此时结构模型失配仍是随机化宽度可能更有价值的场合。总体而言,建议先测量可得参数,再对剩余不确定性进行随机化。
原文摘要 · Abstract (English)
Simulation-to-reality transfer, often called sim-to-real transfer, is a central challenge in robot learning. Yet, the tradeoff between measuring a system more accurately and training over a broader range of simulated dynamics is still poorly understood. In this work, we focused on the allocation of real-robot measurement time between system identification and domain randomization. We studied this tradeoff in a controlled sim-to-sim pendulum setting, where a hidden-parameter model stands in for the physical robot, and the experiment sweeps identification rollouts against the width of the randomization distribution. Across the reality gaps and noise levels we tested, the measurement budget did most of the work. A small number of identification rollouts closed most of the transfer gap, and once any real data was available, policies performed best when trained at the estimated parameters rather than over a widened randomization band. Broad randomization that contained the true system still did not substitute for measurement. These results hold in a benign regime where the dynamics are identifiable and only two parameters are unknown, so structural model mismatch remains the setting where randomization breadth may become more valuable. Overall, our results suggest that sim-to-real pipelines should first measure the parameters they can and reserve randomization for the uncertainty that remains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。