让神经网络自动适应数据变化，通过软重置参数提升持续学习性能
Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
- 用自适应漂移的奥恩斯坦-乌伦贝克过程建模参数变化
- 在非平稳监督与离策略强化学习中表现优于传统方法
- 适合长期运行、数据分布持续变化的场景
传统神经网络训练假设数据来自平稳分布，但实际中分布偏移、持续学习和非平稳上下文老虎机等场景日益普遍。本文提出一种新学习方法，通过带自适应漂移参数的奥恩斯坦-乌伦贝克过程，自动建模并适应非平稳性。该过程使参数趋向初始分布，相当于一种软参数重置。实验表明，该方法在非平稳监督学习和离策略强化学习中表现优异。
原文摘要 · Abstract (English)
Neural networks are traditionally trained under the assumption that data come from a stationary distribution. However, settings which violate this assumption are becoming more popular; examples include supervised learning under distributional shifts, reinforcement learning, continual learning and non-stationary contextual bandits. In this work we introduce a novel learning approach that automatically models and adapts to non-stationarity, via an Ornstein-Uhlenbeck process with an adaptive drift parameter. The adaptive drift tends to draw the parameters towards the initialisation distribution, so the approach can be understood as a form of soft parameter reset. We show empirically that our approach performs well in non-stationary supervised and off-policy reinforcement learning settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。