解决时间序列流中概念漂移与遗忘问题的连续神经网络
cPNN: Continuous Progressive Neural Networks for Evolving Streaming Time Series
- 基于循环网络构建连续渐进结构，持续学习新概念
- 在真实数据流上实现快速适应，对概念漂移保持鲁棒性
- 适合需要长期在线学习的时序预测场景
处理无限数据流需克服数据独立同分布的假设。实际数据流常具时间依赖性（即为时间序列），且数据分布随时间变化（概念漂移）。现有方法分别应对两类问题，缺乏联合解决方案。同时，学习多个概念需记忆旧知识（避免神经网络中的灾难性遗忘）。本文提出连续渐进神经网络（cPNN），能有效应对概念漂移、处理时间依赖，并规避灾难性遗忘。cPNN是渐进神经网络的连续版本，用于保留旧知识并快速迁移至新概念。基于循环神经网络，采用适用于具有时间依赖性的数据流的随机梯度下降。消融实验表明，cPNN能快速适应新概念，对漂移具有强鲁棒性。
原文摘要 · Abstract (English)
Dealing with an unbounded data stream involves overcoming the assumption that data is identically distributed and independent. A data stream can, in fact, exhibit temporal dependencies (i.e., be a time series), and data can change distribution over time (concept drift). The two problems are deeply discussed, and existing solutions address them separately: a joint solution is absent. In addition, learning multiple concepts implies remembering the past (a.k.a. avoiding catastrophic forgetting in Neural Networks' terminology). This work proposes Continuous Progressive Neural Networks (cPNN), a solution that tames concept drifts, handles temporal dependencies, and bypasses catastrophic forgetting. cPNN is a continuous version of Progressive Neural Networks, a methodology for remembering old concepts and transferring past knowledge to fit the new concepts quickly. We base our method on Recurrent Neural Networks and exploit the Stochastic Gradient Descent applied to data streams with temporal dependencies. Results of an ablation study show a quick adaptation of cPNN to new concepts and robustness to drifts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。