在部分可观测环境下，用记忆机制提升零样本强化学习性能。
Zero-Shot Reinforcement Learning Under Partial Observability
- 引入记忆模块应对部分可观测性挑战。
- 在状态、奖励和动态变化均部分可观测时表现优于无记忆基线。
- 适合研究部分可观测强化学习或需零样本泛化的应用者。
近期研究显示，在特定假设下，零样本强化学习（RL）方法可在奖励自由预训练后泛化至任意未见任务。然而，访问马尔可夫状态是关键假设之一，而许多现实应用中该状态仅部分可观测。本文探讨标准零样本RL方法在部分可观测条件下的性能退化问题，并表明与单任务RL类似，基于记忆的架构是有效解决方案。我们在状态、奖励及动态变化均部分可观测的领域评估了所提出的记忆型零样本RL方法，结果表明其性能显著优于无记忆基线。代码已开源：https://enjeeneer.io/projects/bfms-with-memory/。
原文摘要 · Abstract (English)
Recent work has shown that, under certain assumptions, zero-shot reinforcement learning (RL) methods can generalise to any unseen task in an environment after reward-free pre-training. Access to Markov states is one such assumption, yet, in many real-world applications, the Markov state is only partially observable. Here, we explore how the performance of standard zero-shot RL methods degrades when subjected to partially observability, and show that, as in single-task RL, memory-based architectures are an effective remedy. We evaluate our memory-based zero-shot RL methods in domains where the states, rewards and a change in dynamics are partially observed, and show improved performance over memory-free baselines. Our code is open-sourced via: https://enjeeneer.io/projects/bfms-with-memory/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。