用强化学习选最佳示范,缺传感器也能稳控机器人
Reinforcement Learning-Guided Retrieval with Soft Fusion for Robust Multimodal Imitation Learning under Missing Modalities
- 用RL从库中选最相关示范,软融合多示范动作
- 缺摄像头时仍能准确执行,成功率比现有方法高18.3%
- 无需重训练,适合真实场景中传感器不稳定的机器人
机器人通过视觉和自然语言等多模态信号感知环境并执行动作,但实际部署中传感器可能失效或丢失。本文提出RL4IL,一种基于强化学习的模仿学习方法:利用近端策略优化训练的策略,在广度优先搜索候选集中排序示范样本,并通过软交叉注意力融合头生成最终动作。当某模态缺失时,针对各模态的独立强化学习检索策略会从训练库中选取替代示范,再通过软插补头利用顶部候选示范的交叉注意力重建缺失嵌入。该方法无需重新训练策略网络。在三个LIBERO基准测试中,即使在传感器掉线情况下,性能显著优于当前最优方法,平均成功率提升18.3%。
原文摘要 · Abstract (English)
Robotic systems perceive the world through multiple input modalities -- including visual camera streams and natural language instructions -- and must select appropriate actions based on these signals. However, assuming the permanent availability of all input devices is unrealistic, as sensors may fail, become occluded, or drop out entirely during deployment. Robust handling of such missing-modality scenarios is therefore essential for real-world robot operation. This paper introduces RL4IL, a reinforcement learning guided method for imitation learning that selects the most suitable action for a given observation by identifying the most relevant expert demonstrations from a training library. A reinforcement learning policy, trained via Proximal Policy Optimisation over Breadth-First Search candidate sets, ranks candidate demonstrations and a soft cross-attention fusion head aggregates their action signals to produce the final prediction. When a modality is missing at inference time, a dedicated per-modality RL retrieval policy identifies donor demonstrations from the training library, and a soft imputation head reconstructs the missing embedding via cross-attention over the top-ranked donors -- without requiring any retraining of the system. Experiments on three LIBERO benchmark suites demonstrate that RL4IL substantially outperforms state-of-the-art imitation learning methods under sensor dropout conditions, while requiring no policy network training. The code can be found at https://github.com/h-ismkhan/Reinforcement-Learning-via-kNN-for-Robotic-Learning-with-Missing-Camera
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。