不修改模型参数，让机器人实时判断动作成功率并调整策略。
Update-Free On-Policy Steering via Verifiers
- 用初始评估数据训练验证器，实时预测动作成功概率。
- 在5个真实任务中平均提升49%成功率，无需更新基础模型参数。
- 适合希望快速增强现有策略性能的机器人开发者。
近年来，行为克隆（BC）已成为从人类示范中学习操作任务的主要方法。尽管取得成功，BC策略通常脆弱且难以实现精确操作。为此，我们提出无更新在线策略引导方法（UF-OPS），使机器人能在执行时预测动作的成功概率，并动态调整策略。通过在初始评估阶段获取的策略回放数据训练验证器函数，这些验证器随后用于引导基础策略选择更高成功率的动作。该方法在不改变基础参数的前提下，显著提升了黑箱扩散策略的表现，具有轻量且灵活的优势。我们在仿真和真实世界数据上均进行了验证，在5个真实任务中平均成功率达基线的1.49倍。
原文摘要 · Abstract (English)
In recent years, Behavior Cloning (BC) has become one of the most prevalent methods for learning manipulation from human demonstrations. Despite their successes, BC policies are often brittle and struggle with precise manipulation. To overcome these issues, we propose UF-OPS, an Update-Free On-Policy Steering method that enables the robot to predict the success likelihood of its actions and adapt its strategy at execution time. We accomplish this by training verifier functions using policy rollout data obtained during an initial evaluation of the policy. These verifiers are subsequently used to steer the base policy toward actions with a higher likelihood of success. Our method improves the performance of black-box diffusion policies, without changing the base parameters, making it lightweight and flexible. We present results from both simulation and real-world data and achieve an average 49% improvement in success rate over the base policy across 5 real tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。