提出轻量级强化学习方法,让机器人在真实环境中高效自适应导航。
Incremental Residual Reinforcement Learning Toward Real-World Learning for Social Navigation

- 结合增量学习与残差策略,无需经验回放和批量更新。
- 仿真中性能媲美传统方法,真实场景下可快速适应新环境。
- 适合边缘设备部署,适用于复杂多变的真实社交导航任务。
随着移动机器人需求增长,社交导航成为关键任务,推动深度强化学习(RL)研究。然而,行人行为与社交规范在不同地区差异大,仿真难以覆盖所有真实场景。直接在物理环境中进行真实世界强化学习虽有潜力,但面临边缘设备计算资源受限与学习效率低的挑战。本文提出增量残差强化学习(IRRL),融合无需经验回放或批量更新的轻量级增量学习,以及仅对基线策略残差进行训练的残差强化学习,提升效率。仿真结果表明,尽管无回放缓冲区,IRRL性能仍可媲美基于回放的传统方法,并优于现有增量学习方案。真实世界实验进一步验证,IRRL使机器人能有效适应此前未见过的环境。
原文摘要 · Abstract (English)
As the demand for mobile robots continues to increase, social navigation has emerged as a critical task, driving active research into deep reinforcement learning (RL) approaches. However, because pedestrian dynamics and social conventions vary widely across different regions, simulations cannot easily encompass all possible real-world scenarios. Real-world RL, in which agents learn while operating directly in physical environments, presents a promising solution to this issue. Nevertheless, this approach faces significant challenges, particularly regarding constrained computational resources on edge devices and learning efficiency. In this study, we propose incremental residual RL (IRRL). This method integrates incremental learning, which is a lightweight process that operates without a replay buffer or batch updates, with residual RL, which enhances learning efficiency by training only on the residuals relative to a base policy. Through the simulation experiments, we demonstrated that, despite lacking a replay buffer, IRRL achieved performance comparable to those of conventional replay buffer-based methods and outperformed existing incremental learning approaches. Furthermore, the real-world experiments confirmed that IRRL can enable robots to effectively adapt to previously unseen environments through the real-world learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。