Sable让多智能体强化学习在大规模下仍高效稳定，适合复杂场景应用。
Sable: a Performant, Efficient and Scalable Sequence Model for MARL
- 用改进的留存机制处理长时序观测，实现高效记忆推理。
- 6个环境测试中34/45任务超越现有最优方法，千智能体仍保持线性内存增长。
- 性能提升源于高效内存设计，适合高规模多智能体系统部署。
随着多智能体强化学习（MARL）向更复杂、更大规模问题发展，算法需具备强性能、内存高效与可扩展性。本文提出Sable，一种高性能、低内存消耗且可扩展的序列建模方法。Sable通过改进Retentive Networks中的留存机制，实现对多智能体观测的高效计算处理，并支持长时序上下文记忆以进行时间推理。在六个多样化环境的广泛评估中，Sable在45项测试任务中有34项显著优于现有最先进方法。此外，随着智能体数量增加，其性能保持稳定，在超过一千个智能体的环境中仍仅呈现线性内存增长。消融实验进一步验证了其性能提升来源及高效的内存使用特性。
原文摘要 · Abstract (English)
As multi-agent reinforcement learning (MARL) progresses towards solving larger and more complex problems, it becomes increasingly important that algorithms exhibit the key properties of (1) strong performance, (2) memory efficiency, and (3) scalability. In this work, we introduce Sable, a performant, memory-efficient, and scalable sequence modeling approach to MARL. Sable works by adapting the retention mechanism in Retentive Networks (Sun et al., 2023) to achieve computationally efficient processing of multi-agent observations with long context memory for temporal reasoning. Through extensive evaluations across six diverse environments, we demonstrate how Sable is able to significantly outperform existing state-of-the-art methods in a large number of diverse tasks (34 out of 45 tested). Furthermore, Sable maintains performance as we scale the number of agents, handling environments with more than a thousand agents while exhibiting a linear increase in memory usage. Finally, we conduct ablation studies to isolate the source of Sable's performance gains and confirm its efficient computational memory usage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。