用预训练模型提升时间序列增量学习,缓解遗忘问题。
PTMs-TSCIL Pre-Trained Models Based Class-Incremental Learning
- 冻结预训练模型,只微调共享适配器,减少干扰。
- 引入特征漂移补偿网络,提升旧类分类精度1.4%-6.1%。
- 适合需持续学习新类别且无法访问历史数据的场景。
时间序列类增量学习(TSCIL)在平衡稳定性与新知识获取能力方面面临挑战,尤其在无法访问历史数据的真实场景下。尽管预训练模型(PTMs)在视觉和自然语言处理领域表现优异,但其在时间序列领域的应用仍不充分,主要因缺乏大规模时间序列预训练模型。本文首次探索基于预训练模型的时间序列类增量学习(TSCIL)。方法采用冻结的PTM主干网络,仅增量微调共享适配器,并通过知识蒸馏缓解特征漂移。此外,提出特征漂移补偿网络(DCN),采用两阶段训练策略精确建模跨任务的特征空间变化,实现旧类原型在新特征空间中的准确投影。利用经DCN校正的原型进行统一分类器重训练,有效缓解特征漂移与灾难性遗忘。在五个真实数据集上的实验表明,相比现有基于PTM的方法,本方法在所有数据集上最终准确率提升1.4%-6.1%,达到当前最优性能。本工作为TSCIL建立了新范式,为持续学习系统的稳定性-可塑性优化提供新思路。
原文摘要 · Abstract (English)
Class-incremental learning (CIL) for time series data faces critical challenges in balancing stability against catastrophic forgetting and plasticity for new knowledge acquisition, particularly under real-world constraints where historical data access is restricted. While pre-trained models (PTMs) have shown promise in CIL for vision and NLP domains, their potential in time series class-incremental learning (TSCIL) remains underexplored due to the scarcity of large-scale time series pre-trained models. Prompted by the recent emergence of large-scale pre-trained models (PTMs) for time series data, we present the first exploration of PTM-based Time Series Class-Incremental Learning (TSCIL). Our approach leverages frozen PTM backbones coupled with incrementally tuning the shared adapter, preserving generalization capabilities while mitigating feature drift through knowledge distillation. Furthermore, we introduce a Feature Drift Compensation Network (DCN), designed with a novel two-stage training strategy to precisely model feature space transformations across incremental tasks. This allows for accurate projection of old class prototypes into the new feature space. By employing DCN-corrected prototypes, we effectively enhance the unified classifier retraining, mitigating model feature drift and alleviating catastrophic forgetting. Extensive experiments on five real-world datasets demonstrate state-of-the-art performance, with our method yielding final accuracy gains of 1.4%-6.1% across all datasets compared to existing PTM-based approaches. Our work establishes a new paradigm for TSCIL, providing insights into stability-plasticity optimization for continual learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。