用显式记忆记录关键行为,提升机器人社交导航的抗干扰能力。
Advantage-Driven Explicit Memory for Social Navigation

- 引入非参数记忆,用优势信号索引导致失败的关键步骤
- 在模拟训练中实现真实数据反馈,使模型更适应现实中的异常人群行为
- 适合需要持续学习和鲁棒性社交导航的应用场景
当前机器人策略主要通过参数化模仿学习或强化学习训练,行为信息全存储于网络参数中,对表征学习要求过高。本文提出一种新导航智能体,配备非参数记忆,可显式记录导致关键事件(如碰撞)的历史步骤。该设计带来双重优势:一是将部分行为决策外挂至显式记忆,减轻网络负担;二是通过部署时收集测试数据,实现持续学习,增强对分布外(OOD)社交行为的泛化能力。在社交导航任务中,该方法显著提升了对稀疏高代价失败(如人撞)的记忆能力。若在模拟环境中训练,结合逼真渲染与非视觉人群模拟,该机制还能部分缓解模拟到现实的差距,因部分决策基于真实数据。我们将其嵌入循环PPO架构,利用隐藏状态进行记忆检索,以捕捉连续时空动态。通过强化学习的优势信号,有效利用罕见但高影响事件。
原文摘要 · Abstract (English)
Robot policies are predominantly learned with classical parametric variants of imitation learning or RL, where training stores the agent's behavior exclusively in the policy's network parameters, putting a heavy burden on the representation learning algorithm. We propose a new navigation agent equipped with non-parametric memory which explicitly indexes prior steps leading to critical events. The advantages are twofold: first, it allows the policy to outsource some of its behavior into an explicit memory; second, it encourages a form of continual learning by allowing an agent to collect data from its testing episodes during deployment and therefore to better generalize to OOD situations. In the context of social navigation, we show that this improves the agent's capability to retain sparse, high-cost failures, such as human collisions. If the policy is trained in simulation, this also naturally addresses the sim-to-real gap, partially, by basing some of the decision making on real data. We integrate the explicit memory into a recurrent PPO architecture and use hidden states for memory retrieval to capture continuous spatiotemporal dynamics. The goal of exploiting rare, high-impact events is achieved by leveraging the RL agent's advantage signals. We train our agent in simulation with a combination of photorealistic rendering and non-visual crowd simulation and show that the agent is robust with respect to OOD social behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。