700万参数小模型实现带外部变量的实时零样本时间序列预测。
CITRAS-FM: Tiny Time Series Foundation Model for Covariate-Informed Zero-Shot Forecasting
- 基于分块解码器结构，用移位注意力融合已知外部变量信息。
- 在100个任务上达到同类最小模型最优精度，推理速度低于0.1秒。
- 适合资源受限场景下需快速部署且有外部变量的时间序列预测。
预训练时间序列基础模型（TSFM）已在未见目标序列上实现零样本预测。然而，现有模型计算成本高，对多类型变量支持有限，常忽略影响目标变化的外部变量。为此，我们提出CITRAS-FM，一个仅700万参数的微型TSFM，支持单变量、多变量及外部变量感知的零样本预测，并可在实时CPU环境下运行。该模型基于分块、仅解码器的Transformer架构，引入移位注意力机制于跨变量模块中，有效利用整个预测周期内可访问的已知外部变量。此外，为解决外部变量丰富语料稀缺问题，我们提出CovSynth，通过分解目标序列成分合成真实外部变量。在包含100个任务的fev-bench数据集上的实验表明，CITRAS-FM在子1000万参数模型中达到最优零样本准确率，同时实现低于0.1秒的CPU推理时间，兼顾预测精度与实时部署能力。
原文摘要 · Abstract (English)
Pretrained time series foundation models (TSFMs) have enabled zero-shot forecasting on unseen target series. However, existing TSFMs often incur high computational cost and provide limited support for diverse variable types, often failing to account for covariates that exogenously influence target variability. To address these challenges, we propose CITRAS-FM, a tiny 7M-parameter TSFM that supports univariate, multivariate, and covariate-informed zero-shot forecasting with real-time CPU inference. Built on a patch-based, decoder-only Transformer, CITRAS-FM introduces Shifted Attention into the cross-variate module to effectively exploit known covariates accessible throughout the forecast horizon. Moreover, to enable covariate-aware pretraining despite the scarcity of covariate-rich corpora, we propose CovSynth, which synthesizes realistic covariates from decomposed components of target series. Experiments on fev-bench, spanning 100 tasks across various settings, demonstrate that CITRAS-FM achieves state-of-the-art zero-shot accuracy among sub-10M TSFMs while delivering sub-0.1-second CPU inference, offering a strong balance between forecasting accuracy and real-time deployability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。