让机器人优化时自动避开导致坠毁的参数，提升安全性和效率。
Preferential Bayesian Optimization with Crash Feedback
- 通过用户对实验结果的偏好和坠毁反馈，动态调整搜索方向。
- 在合成函数测试中，坠毁率降低63%，数据使用效率更高。
- 适合需要人类反馈的机器人控制、硬件调参等高风险场景。
贝叶斯优化是控制与机器人领域参数学习的常用黑箱优化方法。传统方法依赖可量化的客观目标函数，但在实际应用中，性能指标常因复杂或不可测量而无法获取。偏好型贝叶斯优化（PBO）通过人机对比回馈克服此限制，无需显式量化性能。然而，在硬件系统（如四轴飞行器控制）中，实验坠毁会导致重启耗时、设备损耗等不良后果。标准PBO无法利用坠毁反馈，导致频繁探索高风险参数。为此，本文提出CrashPBO，一种支持用户同时表达偏好与报告坠毁的机制。在合成函数上的基准测试表明，该方法使坠毁率降低63%，并显著提升数据效率。在三个机器人平台的实验证明了CrashPBO的广泛适用性与可迁移性，展现出一种灵活、友好的基于偏好与坠毁反馈的参数学习框架。
原文摘要 · Abstract (English)
Bayesian optimization is a popular black-box optimization method for parameter learning in control and robotics. It typically requires an objective function that reflects the user's optimization goal. However, in practical applications, this objective function is often inaccessible due to complex or unmeasurable performance metrics. Preferential Bayesian optimization (PBO) overcomes this limitation by leveraging human feedback through pairwise comparisons, eliminating the need for explicit performance quantification. When applying PBO to hardware systems, such as in quadcopter control, crashes can cause time-consuming experimental resets, wear and tear, or otherwise undesired outcomes. Standard PBO methods cannot incorporate feedback from such crashed experiments, resulting in the exploration of parameters that frequently lead to experimental crashes. We thus introduce CrashPBO, a user-friendly mechanism that enables users to both express preferences and report crashes during the optimization process. Benchmarking on synthetic functions shows that this mechanism reduces crashes by 63% and increases data efficiency. Through experiments on three robotics platforms, we demonstrate the wide applicability and transferability of CrashPBO, highlighting that it provides a flexible, user-friendly framework for parameter learning with human feedback on preferences and crashes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。