不同神经网络在强化学习中表现各异，选对模型能显著提升效率与效果。
RLBenchNet: The Right Network for the Right Reinforcement Learning Task
- 按任务类型匹配网络：MLP适合可观测连续控制，循环网络应对部分可观测环境。
- Mamba模型吞吐量是LSTM的4.5倍、GRU的3.9倍，性能相当且更高效。
- 处理高记忆需求任务时，仅变压器类和Mamba-2有效，后者内存占用仅为前者的1/8。
强化学习在多种神经网络架构的应用下取得显著进展。本研究系统评估了LSTM、MLP、Mamba/Mamba-2、Transformer-XL、Gated Transformer-XL和GRU在连续控制、离散决策和记忆依赖环境中的表现。结果表明：(1) MLP在完全可观测的连续控制任务中表现最优，兼具性能与效率；(2) LSTM与GRU在部分可观测且记忆需求适中的环境中具有鲁棒性；(3) Mamba模型吞吐量比LSTM高4.5倍、比GRU高3.9倍，性能相当；(4) 仅有Transformer-XL、Gated Transformer-XL和Mamba-2能解决最复杂的记忆密集型任务，其中Mamba-2内存消耗仅为Transformer-XL的1/8。研究为基于任务特性和计算约束选择合适架构提供依据。代码已开源：https://github.com/SafeRL-Lab/RLBenchNet
原文摘要 · Abstract (English)
Reinforcement learning (RL) has seen significant advancements through the application of various neural network architectures. In this study, we systematically investigate the performance of several neural networks in RL tasks, including Long Short-Term Memory (LSTM), Multi-Layer Perceptron (MLP), Mamba/Mamba-2, Transformer-XL, Gated Transformer-XL, and Gated Recurrent Unit (GRU). Through comprehensive evaluation across continuous control, discrete decision-making, and memory-based environments, we identify architecture-specific strengths and limitations. Our results reveal that: (1) MLPs excel in fully observable continuous control tasks, providing an optimal balance of performance and efficiency; (2) recurrent architectures like LSTM and GRU offer robust performance in partially observable environments with moderate memory requirements; (3) Mamba models achieve a 4.5x higher throughput compared to LSTM and a 3.9x increase over GRU, all while maintaining comparable performance; and (4) only Transformer-XL, Gated Transformer-XL, and Mamba-2 successfully solve the most challenging memory-intensive tasks, with Mamba-2 requiring 8x less memory than Transformer-XL. These findings provide insights for researchers and practitioners, enabling more informed architecture selection based on specific task characteristics and computational constraints. Code is available at: https://github.com/SafeRL-Lab/RLBenchNet
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。