用元学习让智能体实时探索新状态，通过记忆密度反馈提升探索效率。
Meta-Learning to Explore via Memory Density Feedback
- 基于元学习，让智能体在单个回合内学会最大化探索进度。
- 通过记忆观测密度变化，减少对已见状态的重复访问。
- 适合复杂动态环境中的强化学习探索任务。
强化学习中的探索算法通常通过引入额外的内在奖励来激励智能体寻找环境中的未见状态。本文提出一种基于元学习（meta-learning）的探索方法，使智能体在单个训练回合内即能学习最大化探索进展，甚至跨越训练周期。该智能体学习一个策略，旨在最小化新观测相对于全部记忆的概率密度。同时，它接收当前观测密度的反馈，并将其存储于循环网络中。通过记住密度变化轨迹，智能体可实时导航不断扩展的熟悉度空间，在完全陌生的状态下仍能有效探索，其策略无需针对这些状态进行训练即可生效。
原文摘要 · Abstract (English)
Exploration algorithms for reinforcement learning typically replace or augment the reward function with an additional ``intrinsic'' reward that trains the agent to seek previously unseen states of the environment. Here, we consider an exploration algorithm that exploits meta-learning, or learning to learn, such that the agent learns to maximize its exploration progress within a single episode, even between epochs of training. The agent learns a policy that aims to minimize the probability density of new observations with respect to all of its memories. In addition, it receives as feedback evaluations of the current observation density and retains that feedback in a recurrent network. By remembering trajectories of density, the agent learns to navigate a complex and growing landscape of familiarity in real-time, allowing it to maximize its exploration progress even in completely novel states of the environment for which its policy has not been trained.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。