让机器人学会避开危险行为,训练后无需额外计算。
Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering

- 通过短时重训练将目标行为信号注入策略权重
- 部署成功率提升44个百分点,零推理开销
- 适配多种机器人模型和真实场景,效果接近重训
行为克隆策略常从演示数据中学习多种行为模式,包括部署时可能不安全的模式(如刀刃朝前传递)。传统方法如数据筛选或推理阶段引导需重新训练或增加推理负担。本文提出MoRE(模式重定向),通过短暂的“去克隆”步骤,将临时模式分类器的引导信号蒸馏进策略权重,实现行为导向。保留损失确保关键能力不丢失,使策略独立运行时即可抑制不良模式,无额外推理开销。在八项模拟与真实任务中,相比原始混合模式策略,平均部署成功率提升44个百分点。在所有对比基线中,MoRE表现最优,逼近过滤数据重训基准,同时保持任务能力和推理速度。该方法适用于多种机器人策略主干(如Diffusion Policy、Pi0.5 VLA)、任务类别及真实部署。
原文摘要 · Abstract (English)
Behavior-cloned policies often learn multiple behavior modes from demonstration datasets, including modes that are unsafe or otherwise undesired at deployment. For example, a policy trained on diverse handover demonstrations may learn to pass a knife blade-first. Standard remedies such as data curation and inference-time steering either require access to the original demonstrations for full retraining or add substantial inference-time overhead. To address this gap, we propose MoRE(Mode Redirection), which redirects policy rollouts toward desired behavior modes through a short "uncloning" step. Specifically, MoRE distills the redirection signal from a temporary mode classifier into the policy weights to steer behavior. A retain loss balances this edit by preserving desired-mode competence, allowing the standalone policy to suppress unwanted modes with zero inference-time overhead. Across eight simulated and real-world tasks, MoRE improves the average deployment success rate (SR) by 44 percentage points over the original mixed-mode policy. Among all compared adaptation and steering baselines, MoRE achieves the strongest SR and approaches the filtered-data retraining reference, while preserving task competence and inference speed. MoRE also generalizes across robot policy backbones, including Diffusion Policy and the Pi0.5 VLA, diverse task categories, and real-world deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。