用持续更新的3D重建和轨迹记忆,让智能体在复杂3D场景中高效探索。
Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration

- 构建持续更新的3D地图,结合序列化视觉记忆,实现长期探索。
- 在HM3D上纯好奇心驱动训练,零样本迁移至Gibson和生成世界。
- 部署仅需RGB图像,可快速适配苹果采摘、图像目标导航等任务。
在稀疏奖励、长时程的3D任务中,探索是学习有效行为的前提。好奇心驱动强化学习通过预测模型与现实的不匹配生成内在奖励来解决此问题。然而,在复杂、逼真的环境中,代理可能陷入局部循环,因重访遗忘状态而获得新奖励。本文表明,根源在于缺乏空间持久性与情景上下文。我们提出使用在线3D重建作为持续更新的世界模型,并将代理策略设计为基于RGB观测的序列模型以保持情景轨迹记忆。该架构在训练中实现高效探索,部署时仅需RGB帧即可导航。在HM3D上纯好奇心训练的代理优于基于RL的主动建图基线,并在零样本条件下泛化至Gibson和AI生成环境。其端到端策略可高效适应下游任务,如苹果采摘和图像目标导航,表现超过从头训练基线。
原文摘要 · Abstract (English)
Exploration is a prerequisite for learning useful behaviors in sparse-reward, long-horizon tasks, particularly within 3D environments. Curiosity-driven reinforcement learning addresses this via intrinsic rewards derived from the mismatch between the agent's predictive model of the world and reality. However, translating this intrinsic motivation to complex, photorealistic environments remains difficult, as agents can become trapped in local loops and receive fresh rewards for revisiting forgotten states. In this work, we demonstrate that this failure stems from a lack of spatial persistence and episodic context. We show that effective curiosity requires a model of the world that is persistent and continuously updated, paired with an agent that maintains an episodic trajectory history to navigate toward novel regions. We achieve this using an online 3D reconstruction as a persistent model of the world, while the agent policy is parameterized as a sequence model over RGB observations to maintain episodic context. This design enables effective exploration during training while allowing the agent to navigate using solely RGB frames at deployment. Trained purely via curiosity on HM3D, our agent outperforms RL-based active mapping baselines and generalizes zero-shot to Gibson and AI-generated worlds. Our end-to-end policy enables efficient adaptation to downstream tasks, such as apple picking and image-goal navigation, outperforming from-scratch baselines. Please see video results at https://recuriosity.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。