用环形吸引子提升强化学习决策速度与精度
Spatial-Aware Decision-Making with Ring Attractors in Reinforcement Learning Systems
- 借鉴神经环路机制，用环形结构编码动作空间
- 在Atari 100k上性能比基线提升53%
- 适合需要稳定探索的机器人与游戏控制场景
环形吸引子是一种受神经回路动力学启发的数学模型，能显著提升强化学习（RL）的学习速度与准确性。作为类脑结构，它可显式编码空间信息与不确定性，在深度强化学习（DRL）中组织神经活动，实现空间表征在神经网络中的分布。该结构还提供时序滤波能力，稳定探索过程中的动作选择，如保持机器人控制中旋转角的连续性或游戏中战术动作的邻接性。本研究通过构建外置模型和将环形吸引子嵌入DRL智能体，实现动作到环上位置的映射及基于神经活动的动作解码。实验表明，该方法在Atari 100k基准上性能较选定基线提升53%。
原文摘要 · Abstract (English)
Ring attractors, mathematical models inspired by neural circuit dynamics, provide a biologically plausible mechanism to improve learning speed and accuracy in Reinforcement Learning (RL). Serving as specialized brain-inspired structures that encode spatial information and uncertainty, ring attractors explicitly encode the action space, facilitate the organization of neural activity, and enable the distribution of spatial representations across the neural network in the context of Deep Reinforcement Learning (DRL). These structures also provide temporal filtering that stabilizes action selection during exploration, for example, by preserving the continuity between rotation angles in robotic control or adjacency between tactical moves in game-like environments. The application of ring attractors in the action selection process involves mapping actions to specific locations on the ring and decoding the selected action based on neural activity. We investigate the application of ring attractors by both building an exogenous model and integrating them as part of DRL agents. Our approach significantly improves state-of-the-art performance on the Atari 100k benchmark, achieving a 53% increase in performance over selected baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。