Transformer可自动拟合多变量时间序列自回归模型，理论与实验验证其有效性。
Transformers and Their Roles as Time Series Foundation Models
- 通过梯度下降让Transformer拟合单变量时间序列的自回归模型。
- MOIRAI模型可自动处理任意数量协变量的多变量时间序列建模。
- 在满足Dobrushin条件时，预训练具备理论泛化边界，适合时间序列基础建模研究者。
本文全面分析了Transformer作为时间序列基础模型的逼近与泛化能力。首先，我们证明存在Transformer可通过梯度下降拟合输入单变量时间序列的自回归模型。随后，我们分析了能够处理任意数量协变量的多变量时间序列基础模型MOIRAI，证明其能自动拟合具有任意数量协变量的自回归模型，揭示了其设计原理与实证成功的原因。在泛化方面，当数据满足Dobrushin条件时，我们建立了预训练的泛化界。实验结果支持理论发现，凸显了Transformer作为时间序列基础模型的有效性。
原文摘要 · Abstract (English)
We give a comprehensive analysis of transformers as time series foundation models, focusing on their approximation and generalization capabilities. First, we demonstrate that there exist transformers that fit an autoregressive model on input univariate time series via gradient descent. We then analyze MOIRAI, a multivariate time series foundation model capable of handling an arbitrary number of covariates. We prove that it is capable of automatically fitting autoregressive models with an arbitrary number of covariates, offering insights into its design and empirical success. For generalization, we establish bounds for pretraining when the data satisfies Dobrushin's condition. Experiments support our theoretical findings, highlighting the efficacy of transformers as time series foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。