用卡尔曼滤波替代奖励归一化，让强化学习更稳定高效。
K-Score: Kalman Filter as a Principled Alternative to Reward Normalization in Reinforcement Learning
- 用一维卡尔曼滤波在线估计奖励均值，动态平滑高方差回报
- 在LunarLander和CartPole上收敛更快，训练方差降低显著
- 无需修改模型结构，计算开销极小，适合快速部署
我们提出一种简单有效的强化学习策略梯度方法，用一维卡尔曼滤波替代传统的奖励归一化。该方法通过递归估计隐藏的奖励均值，有效平滑高方差回报，并适应非平稳环境。相比固定启发式方法，本方案无需修改现有策略架构，计算开销极小。在LunarLander和CartPole上的实验表明，使用卡尔曼滤波后的奖励能显著加速收敛并降低训练方差。代码已公开于https://github.com/Sumxiaa/Kalman_Normalization。
原文摘要 · Abstract (English)
We propose a simple yet effective alternative to reward normalization in policy gradient reinforcement learning by integrating a 1D Kalman filter for online reward estimation. Instead of relying on fixed heuristics, our method recursively estimates the latent reward mean, smoothing high-variance returns and adapting to non-stationary environments. This approach incurs minimal overhead and requires no modification to existing policy architectures. Experiments on \textit{LunarLander} and \textit{CartPole} demonstrate that Kalman-filtered rewards significantly accelerate convergence and reduce training variance compared to standard normalization techniques. Code is available at https://github.com/Sumxiaa/Kalman_Normalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。