MATE用累积嵌入记忆解决上下文MDP问题，高效且性能不输主流模型。
MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddings
- 用累积转移嵌入的求和记忆替代复杂后验推断
- 在多个基准上达到与序列模型相当的性能
- 避免Transformer和RNN的计算或训练缺陷
我们提出MATE，一种针对上下文马尔可夫决策过程（CMDPs）的简单而有效的记忆架构。在CMDPs中，最优智能体可通过维护上下文后验信念在线适应。MATE利用后验的置换不变性，以求和聚合的记忆替代难以计算的后验，保持了理论上充分的表达能力。相比以往记忆架构，MATE避免了Transformer随步数增长的推理开销，以及RNN常见的梯度问题。在多个不同基准上的大量评估表明，MATE在计算效率上具有明显优势，同时性能接近标准序列模型基线。
原文摘要 · Abstract (English)
We propose MATE, a simple yet effective memory architecture for solving Contextual Markov Decision Processes (CMDPs), a family of MDPs parameterized by an unobserved context. In CMDPs, an optimal agent can adapt online by maintaining the posterior belief over contexts. MATE replaces this intractable posterior with a sum-aggregated memory, leveraging the posterior's permutation invariance to retain provably sufficient expressiveness. Compared to prior memory architectures, MATE avoids the growing per-step rollout cost of Transformers and the gradient issues commonly associated with Recurrent Neural Networks (RNNs). Extensive evaluations across diverse benchmarks demonstrate that MATE provides clear computational advantages while achieving performance comparable to standard sequence-model baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。