arXiv:2510.12681cs.LG2025-10被引 5

让时间序列大模型学会利用多源外部信息,提升预测精度。

CoRA: Covariate-Aware Adaptation of Time Series Foundation Models

  • 用因果嵌入自动筛选关键外部变量,动态加权融合。
  • 在真实数据上实现31.1%的均方误差降低,少样本下也有效。
  • 兼容多种大模型,支持时序、文本、图像等多模态输入。

时间序列基础模型(TSFMs)凭借其强大的模型容量、可扩展性及零样本泛化能力展现出显著影响。然而,由于变量间依赖关系异质性以及对大规模多变量数据集的可扩展性限制,多数TSFMs通常仅在单变量时间序列上预训练,因而忽视了实际预测任务中多样外部协变量的关键信息。为此,本文提出一种通用的协变量感知适应框架CoRA,充分利用预训练基础模型的骨干网络,同时有效融合来自时序、语言、图像等多种模态的外生协变量,以提升预测质量。技术上,CoRA保持初始化一致性与参数稳定性;冻结基础模型主干作为特征提取器,实证表明其输出嵌入比原始数据更具信息量。此外,引入新颖的格兰杰因果嵌入(GCE),自动评估各协变量对目标变量的因果可预测性,并通过零初始化的条件注入机制融合加权嵌入,避免灾难性遗忘,逐步整合外部信息。大量实验表明,无论全量或少样本训练,CoRA在协变量感知预测任务中均超越现有先进方法,实现31.1%的均方误差降低。相较于其他适配方法,CoRA具有强兼容性,支持多种先进TSFMs,并将协变量范围拓展至多模态,为TSFMs的实际应用提供了可行范式。

原文摘要 · Abstract (English)

Time Series Foundation Models (TSFMs) have shown significant impact through their model capacity, scalability, and zero-shot generalization. However, due to the heterogeneity of inter-variate dependencies and the backbone scalability on large-scale multivariate datasets, most TSFMs are typically pre-trained on univariate time series. This limitation renders them oblivious to crucial information from diverse covariates in real-world forecasting tasks. To further enhance the performance of TSFMs, we propose a general covariate-aware adaptation (CoRA) framework for TSFMs. It leverages pre-trained backbones of foundation models while effectively incorporating exogenous covariates from various modalities, including time series, language, and images, to improve the quality of predictions. Technically, CoRA maintains the equivalence of initialization and parameter consistency during adaptation. With preserved backbones of foundation models as frozen feature extractors, the outcome embeddings from foundation models are empirically demonstrated more informative than raw data. Further, CoRA employs a novel Granger Causality Embedding (GCE) to automatically evaluate covariates regarding their causal predictability with respect to the target variate. We incorporate these weighted embeddings with a zero-initialized condition-injection mechanism, avoiding catastrophic forgetting of pre-trained foundation models and gradually integrates exogenous information. Extensive experiments show that CoRA of TSFMs surpasses state-of-the-art covariate-aware deep forecasters with full or few-shot training samples, achieving 31.1% MSE reduction on covariate-aware forecasting. Compared to other adaptation methods, CoRA exhibits strong compatibility with various advanced TSFMs and extends the scope of covariates to other modalities, presenting a practical paradigm for the application of TSFMs.

时间序列多模态因果建模大模型适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。