用强化学习动态调参,让滤波器自动适应复杂噪声环境。
Composite Reward Design in PPO-Driven Adaptive Filtering
- 用PPO智能体直接优化滤波系数,奖励函数兼顾信噪比、误差和残差平滑度。
- 在合成信号和真实心电图上均超越经典方法,且能实时运行。
- 适合处理非线性、时变的动态噪声场景,如无线通信和生物信号监测。
无模型的强化学习自适应滤波方法在无线信号通道、生物医学监测和传感器网络等动态非平稳环境中日益受到关注。传统滤波器如LMS、RLS、Wiener和卡尔曼滤波常受限于平稳性假设、精确噪声统计需求或参数调优脆弱性。本文提出一种基于近端策略优化(PPO)的自适应滤波框架,通过复合奖励函数平衡信噪比提升、均方误差降低与残差平滑性。将自适应滤波建模为马尔可夫决策过程,训练PPO智能体根据噪声变化直接调整滤波系数。在包含多种噪声类型的合成非平稳信号上的实验表明,PPO智能体具有超出训练分布的泛化能力。此外,在MIT-BIH噪声压力测试数据库的实测心电图数据上进行分析,该数据受基线漂移、电极运动和肌电伪影污染。所学的PPO策略实现实时推理,在心电图去噪任务中表现略优于强基线方法。结果验证了策略梯度强化学习在非线性、时变动力系统中作为计算高效且灵活的自适应滤波工具的可行性。
原文摘要 · Abstract (English)
Model-free and reinforcement learning-based adaptive filtering methods are gaining traction for denoising in dynamic, non-stationary environments such as wireless signal channels, biomedical monitoring, and sensor networks. Traditional filters such as LMS, RLS, Wiener, and Kalman are often limited by assumptions of stationarity, the need for exact noise statistics, or fragile parameter tuning. This paper proposes an adaptive filtering framework using Proximal Policy Optimization (PPO), guided by a composite reward that balances SNR improvement, MSE reduction, and residual smoothness. We frame adaptive filtering as a Markov decision process and train a PPO agent to adjust filter coefficients directly in response to changing noise. Experiments on synthetic nonstationary signals with diverse noise types show that the PPO agent generalizes beyond its training distribution. Moreover, real-world analysis is made and evaluated on ECG recordings from the MIT-BIH Noise Stress Test Database corrupted by baseline wander, electrode motion, and muscle artifacts. The learned PPO policy achieves real-time inference and slightly outperforms strong classical baselines on ECG denoising. These results demonstrate the viability of policy-gradient reinforcement learning as a computationally efficient and flexible tool for adaptive filtering in nonlinear, time-varying dynamical systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。