用非线性注意力加速强化学习收敛，提升训练效率。
Optimizing Sensory Neurons: Nonlinear Attention Mechanisms for Accelerated Convergence in Permutation-Invariant Neural Networks for Reinforcement Learning
- 对注意力机制的键向量施加非线性变换，增强特征表达能力。
- 在多个RL任务上实现更快收敛，训练时间显著缩短。
- 适合追求高效训练的强化学习研究者与工程应用者。
强化学习（RL）训练通常需要大量计算资源和长时间训练。为应对这一挑战，我们基于先前提出的具有排列不变性感知处理的神经架构，提出一种改进的注意力机制：对键向量（K）施加非线性变换，通过自定义映射函数生成更丰富的表示（K'）。该非线性注意力（NLA）机制提升了注意力层的表征能力，使智能体能够学习更具表现力的特征交互。结果表明，模型实现了显著更快的收敛速度和更高的训练效率，同时性能与基线相当。这些结果凸显了非线性注意力机制在加速强化学习方面的潜力，且不牺牲有效性。
原文摘要 · Abstract (English)
Training reinforcement learning (RL) agents often requires significant computational resources and prolonged training durations. To address this challenge, we build upon prior work that introduced a neural architecture with permutation-invariant sensory processing. We propose a modified attention mechanism that applies a non-linear transformation to the key vectors (K), producing enriched representations (K') through a custom mapping function. This Nonlinear Attention (NLA) mechanism enhances the representational capacity of the attention layer, enabling the agent to learn more expressive feature interactions. As a result, our model achieves significantly faster convergence and improved training efficiency, while maintaining performance on par with the baseline. These results highlight the potential of nonlinear attention mechanisms to accelerate reinforcement learning without sacrificing effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。