arXiv:2410.12593cs.LGcs.AI2024-10ICLR被引 23

提出轻量级持续学习方法,解决动态传感器数据流的模型更新难题

Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting

  • 用提示词池实现模型增量学习,仅需少量可训练参数
  • 在多个真实数据集上显著优于现有方法,保持高效与稳定
  • 适合长期运行的交通、空气质量等实时预测系统

传感设备的广泛应用带来了大量时空预测数据,如交通流量、空气质量与风能预测。尽管时空图神经网络在静态场景中表现优异,但现实中的数据以流式方式持续到达,传感器网络也在不断扩展。这导致两个挑战:重新训练模型效率低下,以及长期历史数据下灾难性遗忘。为此,我们提出一种基于提示调优的连续时空预测方法,遵循“扩展与压缩”两大原则,通过在基线模型中引入持续提示池,利用内存中存储的少量可学习参数,与主模型联合优化,使模型能逐阶段学习流式数据,完成对应时段的预测任务。在多个真实数据集上的实验表明,该方法在有效性、效率和通用性方面均显著优于当前最优基线。

原文摘要 · Abstract (English)

The widespread deployment of sensing devices leads to a surge in data for spatio-temporal forecasting applications such as traffic flow, air quality, and wind energy. Although spatio-temporal graph neural networks have achieved success in modeling various static spatio-temporal forecasting scenarios, real-world spatio-temporal data are typically received in a streaming manner, and the network continuously expands with the installation of new sensors. Thus, spatio-temporal forecasting in streaming scenarios faces dual challenges: the inefficiency of retraining models over newly arrived data and the detrimental effects of catastrophic forgetting over long-term history. To address these challenges, we propose a novel prompt tuning-based continuous forecasting method, following two fundamental tuning principles guided by empirical and theoretical analysis: expand and compress, which effectively resolve the aforementioned problems with lightweight tuning parameters. Specifically, we integrate the base spatio-temporal graph neural network with a continuous prompt pool, utilizing stored prompts (i.e., few learnable parameters) in memory, and jointly optimize them with the base spatio-temporal graph neural network. This method ensures that the model sequentially learns from the spatio-temporal data stream to accomplish tasks for corresponding periods. Extensive experimental results on multiple real-world datasets demonstrate the multi-faceted superiority of our method over the state-of-the-art baselines, including effectiveness, efficiency, universality, etc.

时空预测持续学习图神经网络提示调优

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。