arXiv:2502.21245cs.LG2025-02被引 20

用BERT思想建模时间序列,实现分类/补全/异常检测的通用理解。

TimesBERT: A BERT-Style Foundation Model for Time Series Understanding

  • 借鉴文档多粒度结构,设计时序版BERT架构学习多粒度特征。
  • 在2600亿个时序点上预训练,下游四类任务均达顶尖水平。
  • 适合需要统一处理多种时序任务的研究者和工程师。

时间序列分析在众多场景中至关重要。除预测外,实际任务还包括分类、填补和异常检测,统称为本文中的时间序列理解。尽管GPT风格模型已被视为时序预测的基础模型,但已在自然语言理解中取得突破的BERT风格架构,尚未充分应用于时间序列理解,可能因原始BERT中关键元素丢失所致。受多变量时序与多句文档共享多粒度结构的启发,我们提出TimesBERT,以学习包含时间模式和变量中心特征的通用时序表示。除了自然的掩码建模外,还引入功能标记预测作为并行任务,体现关键的多粒度结构。模型在跨多个领域的2600亿个时间点上进行预训练。利用多粒度表示,TimesBERT在四个典型下游理解任务中表现优于任务特定模型和语言预训练骨干网络,确立其作为时间序列理解的通用基础模型地位。

原文摘要 · Abstract (English)

Time series analysis is crucial in diverse scenarios. Beyond forecasting, considerable real-world tasks are categorized into classification, imputation, and anomaly detection, underscoring different capabilities termed time series understanding in this paper. While GPT-style models have been positioned as foundation models for time series forecasting, the BERT-style architecture, which has made significant advances in natural language understanding, has not been fully unlocked for time series understanding, possibly attributed to the undesirable dropout of essential elements of BERT. In this paper, inspired by the shared multi-granularity structure between multivariate time series and multisentence documents, we design TimesBERT to learn generic representations of time series including temporal patterns and variate-centric characteristics. In addition to a natural adaptation of masked modeling, we propose a parallel task of functional token prediction to embody vital multi-granularity structures. Our model is pre-trained on 260 billion time points across diverse domains. Leveraging multi-granularity representations, TimesBERT achieves state-of-the-art performance across four typical downstream understanding tasks, outperforming task-specific models and language pre-trained backbones, positioning it as a versatile foundation model for time series understanding.

时间序列BERT基础模型多粒度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。