arXiv:2608.26829cs.LGcs.CV2026-08

用多模态增强时间序列变量语义,提升预测精度。

SAGE: Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting

论文配图:SAGE: Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting
图 1 · 摘自论文原文
  • 用CLIP框架融合变量文本、视觉与时序信息,实现端到端建模。
  • 在8个长期预测基准上达到最新最优,优于传统模型。
  • 适合需要融合领域知识的复杂时序预测任务。

时间序列预测模型处理原始数值序列,缺乏领域专家隐含的语义知识,如变量的物理意义、统计特性与时间动态。现有方法分为两类:一类在推理时依赖大语言模型,计算开销大;另一类在数据集层面使用统一文本提示,忽略各变量间语义差异。本文提出SAGE(Seeing and Augmenting with Grounded Encoding),一个基于CLIP的端到端框架,联合建模时序、跨变量、文本与视觉信息。CLIP文本编码器处理频率增强的图像块与变量标记,门控残差路径注入变量特定描述与统计特征。并行地,冻结的CLIP视觉编码器通过仅训练时的对比目标,将生成的时间序列与时间表示对齐。该双重使用CLIP在不引入大语言模型的前提下,提供互补的语义与视觉监督。在八个长期预测基准和M4数据集上,SAGE均达到最先进性能。消融实验验证了多模态对齐与变量级知识带来的互补增益。

原文摘要 · Abstract (English)

Time series forecasting models operate on raw numerical sequences, lacking the semantic knowledge that domain experts implicitly leverage, such as the physical meaning of each variable, its statistical behavior, and its temporal dynamics. Recent efforts to bridge this gap fall into two camps. Some rely on large language models at inference time, which is computationally expensive. Others apply uniform textual prompts at the dataset level, ignoring the heterogeneous semantics across individual variates. We propose SAGE (Seeing and Augmenting with Grounded Encoding), an end-to-end CLIP-based framework that jointly models temporal, cross-variable, textual, and visual information. The CLIP text encoder processes frequency-enhanced patches and variable tokens, while gated residual paths inject variable-specific descriptions and statistical descriptors. In parallel, the frozen CLIP vision encoder aligns rendered series with temporal representations through a training-only contrastive objective. This dual use of CLIP adds complementary semantic and visual supervision without placing an LLM in the forecasting loop. Across eight long-term benchmarks and M4, SAGE achieves state-of-the-art accuracy. Ablations confirm complementary gains from multimodal alignment and variable-level knowledge.

时序预测多模态语义增强CLIP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。