不改模型权重,运行时实时修正无人机导航行为,大幅降低碰撞率。
Latent Activation Editing: Inference-Time Refinement of Learned Policies for Safer Multirobot Navigation
- 通过检测中间激活状态,识别高风险行为并实时干预修正。
- 实测碰撞减少近90%,多数轨迹实现无碰撞,任务完成率不变。
- 适合部署后优化安全性能,对硬件资源要求低,适用于真实机器人。
强化学习在多旋翼无人机协同导航等复杂领域取得显著进展,但即使训练良好的策略仍可能在障碍物密集环境中发生碰撞。通过重新训练或微调来修复这些罕见但关键的安全问题成本高昂,且可能损害已习得技能。受大语言模型激活控制和计算机视觉潜在编辑的启发,我们提出一种推理时的潜在激活编辑(LAE)框架,可在不修改模型权重或结构的前提下,动态优化预训练策略的行为。该框架分两阶段运行:(i) 在线分类器监测中间激活,识别与不良行为相关状态;(ii) 激活编辑模块对标记激活进行选择性调整,引导策略转向更安全状态。本研究聚焦于多无人机导航安全提升,假设增强策略对风险的内部感知可诱导更安全行为。为此,我们构建了一个预测未来碰撞前激活的潜在碰撞世界模型,促使系统提前、更谨慎地规避。大量仿真与真实世界Crazyflie实验表明,LAE在统计学上显著减少碰撞(相比未编辑基线减少近90%的累计碰撞),大幅提高无碰撞轨迹比例,同时保持任务完成率。结果表明,LAE是一种轻量级、可在资源受限硬件上运行的部署后策略优化范式。
原文摘要 · Abstract (English)
Reinforcement learning has enabled significant progress in complex domains such as coordinating and navigating multiple quadrotors. However, even well-trained policies remain vulnerable to collisions in obstacle-rich environments. Addressing these infrequent but critical safety failures through retraining or fine-tuning is costly and risks degrading previously learned skills. Inspired by activation steering in large language models and latent editing in computer vision, we introduce a framework for inference-time Latent Activation Editing (LAE) that refines the behavior of pre-trained policies without modifying their weights or architecture. The framework operates in two stages: (i) an online classifier monitors intermediate activations to detect states associated with undesired behaviors, and (ii) an activation editing module that selectively modifies flagged activations to shift the policy towards safer regimes. In this work, we focus on improving safety in multi-quadrotor navigation. We hypothesize that amplifying a policy's internal perception of risk can induce safer behaviors. We instantiate this idea through a latent collision world model trained to predict future pre-collision activations, thereby prompting earlier and more cautious avoidance responses. Extensive simulations and real-world Crazyflie experiments demonstrate that LAE achieves statistically significant reduction in collisions (nearly 90% fewer cumulative collisions compared to the unedited baseline) and substantially increases the fraction of collision-free trajectories, while preserving task completion. More broadly, our results establish LAE as a lightweight paradigm, feasible on resource-constrained hardware, for post-deployment refinement of learned robot policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。