arXiv:2410.08421cs.LG2024-10被引 2

将时间序列视为函数,用渐进恢复法提升模型泛化能力

Generalizable autoregressive modeling of time series through functional narratives

  • 把时间序列看作函数,通过降维操作生成渐进简化序列
  • 在合成数据上实现26%性能提升,在真实数据上最高超其他方法6%
  • 适合构建通用时间序列基础模型,可无缝接入现有Transformer架构

时间序列本质上是时间的函数,但现有Transformer多将其视为时间片段的拼接,忽略了函数特性。本文提出一种新目标函数,将时间序列重新诠释为时序函数。通过在函数空间构造不同强度的退化算子,生成原始样本的抽象或简化版本构成新序列。基于此序列,训练自回归Transformer逐步从最简版本恢复原样本。类比语言中预测下一个词以学习叙事,该模型旨在学习时间序列的函数叙事(NoTS)。理论上,该序列更优地逼近函数,相比时间片段序列能处理更多函数类型(如微分),在合成特征回归实验中性能提升26%。实验证明,NoTS在22个真实数据集上的3项任务中均显著优于其他预训练方法,最高提升6%;且可与现有Transformer架构结合持续增益。结果表明,NoTS具有成为通用动态学习器的潜力,为时间序列基础模型提供可行路径。

原文摘要 · Abstract (English)

Time series data are inherently functions of time, yet current transformers often learn time series by modeling them as mere concatenations of time periods, overlooking their functional properties. In this work, we propose a novel objective for transformers that learn time series by re-interpreting them as temporal functions. We build an alternative sequence of time series by constructing degradation operators of different intensity in the functional space, creating augmented variants of the original sample that are abstracted or simplified to different degrees. Based on the new set of generated sequence, we train an autoregressive transformer that progressively recovers the original sample from the most simplified variant. Analogous to the next word prediction task in languages that learns narratives by connecting different words, our autoregressive transformer aims to learn the Narratives of Time Series (NoTS) by connecting different functions in time. Theoretically, we justify the construction of the alternative sequence through its advantages in approximating functions. When learning time series data with transformers, constructing sequences of temporal functions allows for a broader class of approximable functions (e.g., differentiation) compared to sequences of time periods, leading to a 26\% performance improvement in synthetic feature regression experiments. Experimentally, we validate NoTS in 3 different tasks across 22 real-world datasets, where we show that NoTS significantly outperforms other pre-training methods by up to 6\%. Additionally, combining NoTS on top of existing transformer architectures can consistently boost the performance. Our results demonstrate the potential of NoTS as a general-purpose dynamic learner, offering a viable alternative for developing foundation models for time series analysis.

时间序列自回归函数建模Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。