TimeDiT用扩散模型处理时间序列,兼顾不确定性与物理约束。
TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model
- 结合Transformer与扩散模型,学习时间依赖与概率分布。
- 零样本/微调皆可完成预测、填补、异常检测等任务。
- 无需微调即可插入外部知识,适合多场景时间序列应用。
基础模型(如大语言模型)在文本和视频领域取得突破,但时间序列因缺失值、多分辨率等特性面临独特挑战。现有自回归Transformer常忽略不确定性且难融入物理约束。本文提出TimeDiT,一种融合Transformer时序建模与扩散概率采样的通用时间序列扩散变换器。其采用统一掩码机制,使训练与推理在多种任务间一致,并提出理论支持的、免微调的模型编辑策略,实现采样时灵活引入外部知识。系统评估表明,TimeDiT在零样本/微调下均有效完成预测、填补等基础任务,以及多分辨率预测、异常检测、数据生成等域任务,成为连接通用与专用模型的‘原型基础模型’。
原文摘要 · Abstract (English)
Foundation models, particularly Large Language Models (LLMs), have revolutionized text and video processing, yet time series data presents distinct challenges for such approaches due to domain-specific features such as missing values, multi-resolution characteristics, etc. Furthermore, the de-facto autoregressive transformers tend to learn deterministic temporal dependencies within pre-trained data while overlooking inherent uncertainties and lacking integration of physical constraints. In this paper, we introduce TimeDiT, a diffusion transformer model that synergistically combines transformer-based temporal dependency learning with diffusion-based probabilistic sampling. TimeDiT employs a unified masking mechanism to harmonize the training and inference process across diverse tasks while introducing a theoretically grounded, finetuning-free model editing strategy that enables flexible integration of external knowledge during sampling. Acknowledging the challenges of unifying multiple downstream tasks under a single model, our systematic evaluation demonstrates TimeDiT's effectiveness both in fundamental tasks, i.e., forecasting and imputation, through zero-shot/fine-tuning; and in domain tasks, i.e., multi-resolution forecasting, anomaly detection, and data generation, establishing it as a \textit{proto-foundation model} that bridges the gap between general-purpose and domain-specific models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。