arXiv:2506.20553cs.RO2025-06被引 10

用仿真数据降低真实测试成本,提升评估效率

Sim2Val: Leveraging Correlation Across Test Platforms for Variance-Reduced Metric Estimation

  • 利用仿真与真实数据相关性,通过控制变量法减少评估方差
  • 实测样本减少70%以上即可达到相同置信度的性能保证
  • 适合需高可靠性验证的自动驾驶与四足机器人系统

基于学习的机器人系统需要严格的验证以确保可靠性能,但大规模真实世界测试往往成本过高,且可能数据不足。本文提出Sim2Val框架,利用仿真与真实场景配对数据(如仿真输出与真实观测),通过控制变量法提高真实性能指标估计的准确性。将廉价且丰富的辅助测量(如模拟器输出)作为控制变量,可严格降低蒙特卡洛估计的方差,显著减少达成指定置信度所需的真实测试样本量。我们提供了理论分析,证明其方差和采样效率优势,并在自动驾驶与四足机器人任务中实证验证:本方法可在高概率下实现性能边界,且采样效率大幅提升。该技术能有效减轻机器人系统性能验证的真实测试负担,支持更高效、低成本的实验评估。

原文摘要 · Abstract (English)

Learning-based robotic systems demand rigorous validation to assure reliable performance, but extensive real-world testing is often prohibitively expensive, and if conducted may still yield insufficient data for high-confidence guarantees. In this work we introduce Sim2Val, a general estimation framework that leverages paired data across test platforms, e.g., paired simulation and real-world observations, to achieve better estimates of real-world metrics via the method of control variates. By incorporating cheap and abundant auxiliary measurements (for example, simulator outputs) as control variates for costly real-world samples, our method provably reduces the variance of Monte Carlo estimates and thus requires significantly fewer real-world samples to attain a specified confidence bound on the mean performance. We provide theoretical analysis characterizing the variance and sample-efficiency improvement, and demonstrate empirically in autonomous driving and quadruped robotics settings that our approach achieves high-probability bounds with markedly improved sample efficiency. Our technique can lower the real-world testing burden for validating the performance of the stack, thereby enabling more efficient and cost-effective experimental evaluation of robotic systems.

机器人验证控制变量仿真-真实采样效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。