用神经网络直接学习强化学习中的主特征向量，提升效率与可扩展性。
DROGO: Default Representation Objective via Graph Optimization in Reinforcement Learning
- 设计新目标函数，直接用神经网络逼近默认表示的主特征向量。
- 在多个环境中验证有效，显著降低计算开销并支持高维空间。
- 适合需要高效探索与奖励塑形的强化学习研究者使用。
在强化学习中，默认表示（DR）及其主特征向量已被证明在奖励塑造、计数式探索、选项发现和迁移学习等多种任务中表现优异。然而，以往方法需先近似构造DR矩阵，再进行特征分解，计算成本高，难以拓展到高维空间。本文提出一种新目标，可直接通过神经网络逼近DR的主特征向量。实验表明该方法在多个环境上均有效，并成功应用于奖励塑造任务，显著提升计算效率。
原文摘要 · Abstract (English)
In computational reinforcement learning, the default representation (DR) and its principal eigenvector have been shown to be effective for a wide variety of applications, including reward shaping, count-based exploration, option discovery, and transfer. However, in prior investigations, the eigenvectors of the DR were computed by first approximating the DR matrix, and then performing an eigendecomposition. This procedure is computationally expensive and does not scale to high-dimensional spaces. In this paper, we derive an objective for directly approximating the principal eigenvector of the DR with a neural network. We empirically demonstrate the effectiveness of the objective in a number of environments, and apply the learned eigenvectors for reward shaping.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。