arXiv:2410.05711cs.LG2024-10ICML被引 21

TimeDART统一建模时间序列的全局趋势与局部细节,提升自监督表征能力。

TimeDART: A Diffusion Autoregressive Transformer for Self-Supervised Time Series Representation

  • 用因果Transformer+分块嵌入捕捉时间序列演变趋势
  • 引入去噪扩散过程学习细微局部模式,提升表征精度
  • 适用于需要高质量表征的时序预测与分类任务

自监督学习在时间序列分析中日益受到关注,可有效支持多种下游任务并减少对标注数据的依赖。然而,现有方法难以统一建模长期动态演化与细微局部模式。本文提出TimeDART,一种新型自监督时间序列预训练框架,融合两种强大生成范式,以学习更具迁移性的表征。首先,采用因果Transformer编码器结合分块嵌入策略,从左到右建模演化趋势;在此基础上,进一步引入去噪扩散过程,通过前向扩散与反向去噪捕获细粒度局部模式。最后,模型以自回归方式优化。实验表明,TimeDART在多个公开数据集上的时序预测与分类任务中均显著优于现有方法,验证了其有效性。代码已开源。

原文摘要 · Abstract (English)

Self-supervised learning has garnered increasing attention in time series analysis for benefiting various downstream tasks and reducing reliance on labeled data. Despite its effectiveness, existing methods often struggle to comprehensively capture both long-term dynamic evolution and subtle local patterns in a unified manner. In this work, we propose \textbf{TimeDART}, a novel self-supervised time series pre-training framework that unifies two powerful generative paradigms to learn more transferable representations. Specifically, we first employ a causal Transformer encoder, accompanied by a patch-based embedding strategy, to model the evolving trends from left to right. Building on this global modeling, we further introduce a denoising diffusion process to capture fine-grained local patterns through forward diffusion and reverse denoising. Finally, we optimize the model in an autoregressive manner. As a result, TimeDART effectively accounts for both global and local sequence features in a coherent way. We conduct extensive experiments on public datasets for time series forecasting and classification. The experimental results demonstrate that TimeDART consistently outperforms previous compared methods, validating the effectiveness of our approach. Our code is available at https://github.com/Melmaphother/TimeDART.

时间序列自监督扩散模型Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。