通过演示学习记忆机制，让智能体像专家一样记住关键信息做决策。
Learning Memory Mechanisms for Decision Making through Demonstrations
- 用记忆依赖对捕捉专家在何时回忆什么信息来决策
- 在记忆训练任务中性能显著优于标准Transformer
- 适合需要长期记忆的强化学习场景
在部分可观测马尔可夫决策过程（POMDP）中，将智能体的历史信息融入记忆是决策的关键挑战。传统模仿学习依赖观测-动作对，无法捕捉专家决策时的记忆机制。为此，本文提出记忆依赖对（p, q），表示时间p的事件被用于时间q的决策。引入AttentionTuner模型，利用记忆依赖对在Transformer中建模记忆过程，在Memory Gym和Long-term Memory Benchmark上均取得显著提升。代码已开源。
原文摘要 · Abstract (English)
In Partially Observable Markov Decision Processes, integrating an agent's history into memory poses a significant challenge for decision-making. Traditional imitation learning, relying on observation-action pairs for expert demonstrations, fails to capture the expert's memory mechanisms used in decision-making. To capture memory processes as demonstrations, we introduce the concept of memory dependency pairs $(p, q)$ indicating that events at time $p$ are recalled for decision-making at time $q$. We introduce AttentionTuner to leverage memory dependency pairs in Transformers and find significant improvements across several tasks compared to standard Transformers when evaluated on Memory Gym and the Long-term Memory Benchmark. Code is available at https://github.com/WilliamYue37/AttentionTuner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。