用高维掩码机制加速强化学习训练，比传统方法快两倍以上。
A Method of Selective Attention for Reservoir Based Agents
- 设计高维输入掩码模块，动态抑制无关输入信号。
- 在相同条件下，训练速度比无抑制快四倍，比层归一化快两倍。
- 适合需要快速收敛的复杂环境强化学习任务。
深度强化学习代理的训练因无关输入维度的存在而显著变慢。现有模块如层归一化可通过权重衰减实现选择性注意力，即输入掩码，缩小冗余输入的尺度，从而加速策略训练。然而，我们发现增加输入掩码计算中的参数数量可带来更显著的加速效果。一个简单的高维掩码模块与层归一化及无任何输入抑制的模型进行对比，结果显示该高维掩码使训练速度相比无抑制情况提升四倍，相比层归一化提升两倍。
原文摘要 · Abstract (English)
Training of deep reinforcement learning agents is slowed considerably by the presence of input dimensions that do not usefully condition the reward function. Existing modules such as layer normalization can be trained with weight decay to act as a form of selective attention, i.e. an input mask, that shrinks the scale of unnecessary inputs, which in turn accelerates training of the policy. However, we find a surprising result that adding numerous parameters to the computation of the input mask results in much faster training. A simple, high dimensional masking module is compared with layer normalization and a model without any input suppression. The high dimensional mask resulted in a four-fold speedup in training over the null hypothesis and a two-fold speedup in training over the layer normalization method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。