用视觉语言模型融合时序图像与文本，提升少样本预测能力。
Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting
- 通过记忆库检索增强时序特征提取
- 将时间序列转为图像并生成描述性文本
- 在少样本场景下表现优于传统方法
近期时序预测研究尝试引入文本或视觉模态以提升精度。文本虽具上下文理解能力，但缺乏精细时序细节；视觉能捕捉复杂时序模式，却缺少语义信息，制约了模态互补潜力。为此，我们提出 Time-VLM，一种利用预训练视觉-语言模型（VLMs）融合时序、视觉与文本模态的新框架。该框架包含三个核心组件：(1) 检索增强学习器，通过记忆库交互提取丰富时序特征；(2) 视觉增强学习器，将时间序列编码为信息丰富的图像；(3) 文本增强学习器，生成上下文描述文本。三者协同冻结的预训练 VLMs 生成多模态嵌入，并与时序特征融合进行最终预测。大量实验表明，Time-VLM 在少样本与零样本场景下均取得优异性能，为多模态时序预测开辟新方向。代码已开源。
原文摘要 · Abstract (English)
Recent advancements in time series forecasting have explored augmenting models with text or vision modalities to improve accuracy. While text provides contextual understanding, it often lacks fine-grained temporal details. Conversely, vision captures intricate temporal patterns but lacks semantic context, limiting the complementary potential of these modalities. To address this, we propose \method, a novel multimodal framework that leverages pre-trained Vision-Language Models (VLMs) to bridge temporal, visual, and textual modalities for enhanced forecasting. Our framework comprises three key components: (1) a Retrieval-Augmented Learner, which extracts enriched temporal features through memory bank interactions; (2) a Vision-Augmented Learner, which encodes time series as informative images; and (3) a Text-Augmented Learner, which generates contextual textual descriptions. These components collaborate with frozen pre-trained VLMs to produce multimodal embeddings, which are then fused with temporal features for final prediction. Extensive experiments demonstrate that Time-VLM achieves superior performance, particularly in few-shot and zero-shot scenarios, thereby establishing a new direction for multimodal time series forecasting. Code is available at https://github.com/CityMind-Lab/ICML25-TimeVLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。