训练时逐步增长神经网络，提升强化学习模型性能。
Growing with Experience: Growing Neural Networks in Deep Reinforcement Learning
- 训练初期用小网络学基础策略，再渐进添加层
- 在MiniHack和Mujoco上性能比同规模固定网络高最多72%
- 可无缝接入现有强化学习算法，适合复杂策略学习
尽管大型模型已革新机器学习领域，但训练中等规模的神经网络在强化学习中仍面临挑战，严重限制了可学习策略的复杂度。为在保持可训练性的前提下提升网络容量，我们提出GrowNN：一种在训练过程中逐步增长网络的简单有效方法。从一个小网络开始学习初始策略，随后无损地增加层数，后续更新可利用新增层学习更复杂的策略，随着策略复杂度提升逐步扩容。GrowNN可无缝集成到大多数现有强化学习智能体中。在MiniHack和Mujoco上的实验表明，渐进式增长的网络相比同规模静态网络，性能提升最高达48%（MiniHack Room）和72%（Ant）。
原文摘要 · Abstract (English)
While increasingly large models have revolutionized much of the machine learning landscape, training even mid-sized networks for Reinforcement Learning (RL) is still proving to be a struggle. This, however, severely limits the complexity of policies we are able to learn. To enable increased network capacity while maintaining network trainability, we propose GrowNN, a simple yet effective method that utilizes progressive network growth during training. We start training a small network to learn an initial policy. Then we add layers without changing the encoded function. Subsequent updates can utilize the added layers to learn a more expressive policy, adding capacity as the policy's complexity increases. GrowNN can be seamlessly integrated into most existing RL agents. Our experiments on MiniHack and Mujoco show improved agent performance, with incrementally GrowNN-deeper networks outperforming their respective static counterparts of the same size by up to 48% on MiniHack Room and 72% on Ant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。