通过神经网络权重预测强化学习泛化能力并优化训练方法。
Leveraging weights signals -- Predicting and improving generalizability in reinforcement learning
- 用模型内部权重预测强化学习的泛化性能。
- 改进PPO损失函数后，新算法泛化能力更强。
- 适合关注RL泛化性提升的研究者与工程师。
强化学习智能体的泛化能力（在不同于训练环境的场景中表现的能力）是关键问题，因为智能体容易过拟合于训练环境。为解决此问题并提升强化学习智能体的泛化能力，本文提出一种基于智能体神经网络内部权重的新方法，用于预测其泛化能力得分。利用该预测能力，我们对近端策略优化（Proximal Policy Optimization, PPO）的损失函数进行了改进，以增强训练出的智能体的泛化性能。实验结果表明，使用改进后的PPO算法训练的智能体，在多种未见环境中的表现显著优于原始版本，验证了所提方法的有效性。
原文摘要 · Abstract (English)
Generalizability of Reinforcement Learning (RL) agents (ability to perform on environments different from the ones they have been trained on) is a key problem as agents have the tendency to overfit to their training environments. In order to address this problem and offer a solution to increase the generalizability of RL agents, we introduce a new methodology to predict the generalizability score of RL agents based on the internal weights of the agent's neural networks. Using this prediction capability, we propose some changes in the Proximal Policy Optimization (PPO) loss function to boost the generalization score of the agents trained with this upgraded version. Experimental results demonstrate that our improved PPO algorithm yields agents with stronger generalizability compared to the original version.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。