综述时间序列的Transformer基础模型,梳理分类体系与发展方向。
Foundation Models for Time Series: A Survey
- 按架构分为基于补丁和原始序列两类,区分单变量与多变量处理能力。
- 涵盖概率与确定性预测、轻量与大规模模型,揭示性能差异。
- 首次按训练目标函数分类,为模型选型与研究提供系统指引。
基于Transformer的基础模型已成为时间序列分析的主流范式,在预测、异常检测、分类、趋势分析等任务中展现出前所未有的能力。本综述全面梳理当前预训练基础模型的最新进展,提出一种新型分类体系,从多个维度进行归类。具体包括:按架构设计区分基于补丁表示与直接处理原始序列的模型;是否提供概率或确定性预测;是否原生支持单变量或多变量时间序列。此外,还涵盖模型规模与复杂度,突出轻量级模型与大规模基础模型的差异。本综述独特之处在于按训练阶段所采用的目标函数进行分类。综合上述视角,该工作为研究人员与实践者提供系统参考,揭示当前趋势并指明未来研究方向。
原文摘要 · Abstract (English)
Transformer-based foundation models have emerged as a dominant paradigm in time series analysis, offering unprecedented capabilities in tasks such as forecasting, anomaly detection, classification, trend analysis and many more time series analytical tasks. This survey provides a comprehensive overview of the current state of the art pre-trained foundation models, introducing a novel taxonomy to categorize them across several dimensions. Specifically, we classify models by their architecture design, distinguishing between those leveraging patch-based representations and those operating directly on raw sequences. The taxonomy further includes whether the models provide probabilistic or deterministic predictions, and whether they are designed to work with univariate time series or can handle multivariate time series out of the box. Additionally, the taxonomy encompasses model scale and complexity, highlighting differences between lightweight architectures and large-scale foundation models. A unique aspect of this survey is its categorization by the type of objective function employed during training phase. By synthesizing these perspectives, this survey serves as a resource for researchers and practitioners, providing insights into current trends and identifying promising directions for future research in transformer-based time series modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。