arXiv:2601.03683cs.LGcs.NE2026-01

用强化学习优化RNN,让模型自己选关键时间点预测更准

Rethinking Recurrent Neural Networks for Time Series Forecasting: A Reinforced Recurrent Encoder with Prediction-Oriented Proximal Policy Optimization

  • 把RNN内部调整变成决策过程,自动选择重要时间步和隐藏状态
  • 在5个真实数据集上超越现有方法,甚至超过顶尖Transformer模型
  • 适合需要高精度时序预测的工程系统,如电力、交通调度

时序预测在现代工程信息系统中至关重要,递归神经网络(RNN)因其建模序列数据的能力被广泛应用。传统RNN预测器采用仅编码器策略,以滑动历史窗口为输入,但对所有时间步和隐藏状态一视同仁,忽略了其对预测的不同贡献,导致性能不佳。为此,我们提出一种新型强化递归编码器与面向预测的近端策略优化方法(RRE-PPO4Pred),显著提升RNN的建模能力和预测精度。核心创新包括:(1)提出强化递归编码器(RRE)框架,将RNN内部自适应建模为马尔可夫决策过程,构建统一决策环境,实现输入特征选择、隐藏跳连和输出目标选择的联合学习;(2)改进预测导向的近端策略优化算法(PPO4Pred),引入基于Transformer的智能体进行时序推理,并设计动态转移采样策略以提升采样效率;(3)采用共进化优化范式,促进RNN预测器与策略智能体的协同学习,实现自适应、交互式的时序建模。在五个真实世界数据集上的全面评估表明,该方法持续优于现有基线,且预测精度超过当前最先进Transformer模型,为工程信息学提供了先进的时序预测工具。

原文摘要 · Abstract (English)

Time series forecasting plays a crucial role in contemporary engineering information systems for supporting decision-making across various industries, where Recurrent Neural Networks (RNNs) have been widely adopted due to their capability in modeling sequential data. Conventional RNN-based predictors adopt an encoder-only strategy with sliding historical windows as inputs to forecast future values. However, this approach treats all time steps and hidden states equally without considering their distinct contributions to forecasting, leading to suboptimal performance. To address this limitation, we propose a novel Reinforced Recurrent Encoder with Prediction-oriented Proximal Policy Optimization, RRE-PPO4Pred, which significantly improves time series modeling capacity and forecasting accuracy of the RNN models. The core innovations of this method are: (1) A novel Reinforced Recurrent Encoder (RRE) framework that enhances RNNs by formulating their internal adaptation as a Markov Decision Process, creating a unified decision environment capable of learning input feature selection, hidden skip connection, and output target selection; (2) An improved Prediction-oriented Proximal Policy Optimization algorithm, termed PPO4Pred, which is equipped with a Transformer-based agent for temporal reasoning and develops a dynamic transition sampling strategy to enhance sampling efficiency; (3) A co-evolutionary optimization paradigm to facilitate the learning of the RNN predictor and the policy agent, providing adaptive and interactive time series modeling. Comprehensive evaluations on five real-world datasets indicate that our method consistently outperforms existing baselines, and attains accuracy better than state-of-the-art Transformer models, thus providing an advanced time series predictor in engineering informatics.

时序预测强化学习RNN改进工程信息

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。