用图神经网络增强决策变换器，高效解决动态环境下的优化问题
GNN-DT: Graph Neural Network Enhanced Decision Transformer for Efficient Optimization in Dynamic Environments
- 将图神经网络嵌入决策变换器，通过残差连接处理动态状态与动作空间
- 在电动汽车充电优化中仅需更少训练轨迹即达高精度，样本效率显著提升
- 对未知环境和更大动作空间具有强泛化能力，适合复杂实时优化场景
针对现实世界优化问题中常见的动态状态-动作空间、大规模及稀疏奖励等挑战，本文提出GNN-DT——一种融合图神经网络嵌入器与输入输出令牌间新型残差连接的决策变换器架构。该模型通过学习历史轨迹，克服在线强化学习中稀疏奖励的局限，实现实时高质量求解。在复杂的电动汽车充电优化任务上，GNN-DT表现优于现有决策变换器与离线强化学习基线，所需训练轨迹显著减少，样本效率更高。此外，其在未见环境与更大动作空间下仍保持稳健泛化能力，填补了以往离线与在线强化学习方法的关键空白。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) methods used for solving real-world optimization problems often involve dynamic state-action spaces, larger scale, and sparse rewards, leading to significant challenges in convergence, scalability, and efficient exploration of the solution space. This study introduces GNN-DT, a novel Decision Transformer (DT) architecture that integrates Graph Neural Network (GNN) embedders with a novel residual connection between input and output tokens crucial for handling dynamic environments. By learning from previously collected trajectories, GNN-DT tackles the sparse rewards limitations of online RL algorithms and delivers high-quality solutions in real-time. We evaluate GNN-DT on the complex electric vehicle (EV) charging optimization problem and prove that its performance is superior and requires significantly fewer training trajectories, thus improving sample efficiency compared to existing DT and offline RL baselines. Furthermore, GNN-DT exhibits robust generalization to unseen environments and larger action spaces, addressing a critical gap in prior offline and online RL approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。