arXiv:2602.21693cs.LG2026-02被引 1

用多模态专家混合提升时间序列预测,让大模型推理指导数值预测。

TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts

  • 引入多模态专家混合模块,无需对齐即可融合文本与时间序列数据。
  • 在16个真实场景中超越主流方法,显著提升预测准确率。
  • 适合需要解释性和多源信息融合的金融、气象等预测任务。

多模态时间序列预测因能利用其他模态中的丰富信息而受到广泛关注,有望优于传统单模态模型。然而,由于模态对齐的根本挑战,现有方法难以有效融入多模态数据,尤其是对时间序列波动具有因果影响的文本信息(如应急报告和政策公告)。本文反思了文本信息在数值预测中的作用,提出时间序列变压器的多模态专家混合模型(TiMi),以激发大语言模型(LLM)的因果推理能力。具体地,TiMi利用LLM生成对未来发展的推断,作为时间序列预测的引导。为无缝整合外生因素与时间序列数据,我们引入轻量级多模态专家混合(MMoE)模块,作为即插即用组件赋能基于Transformer的时间序列模型进行多模态预测,避免显式表示层面的对齐需求。实验表明,所提方法在16个真实世界多模态预测基准上均达到一致的最先进性能,优于多个先进基线,同时具备强适应性与可解释性。

原文摘要 · Abstract (English)

Multimodal time series forecasting has garnered significant attention for its potential to provide more accurate predictions than traditional single-modality models by leveraging rich information inherent in other modalities. However, due to fundamental challenges in modality alignment, existing methods often struggle to effectively incorporate multimodal data into predictions, particularly textual information that has a causal influence on time series fluctuations, such as emergency reports and policy announcements. In this paper, we reflect on the role of textual information in numerical forecasting and propose Time series transformers with Multimodal Mixture-of-Experts, TiMi, to unleash the causal reasoning capabilities of LLMs. Concretely, TiMi utilizes LLMs to generate inferences on future developments, which serve as guidance for time series forecasting. To seamlessly integrate both exogenous factors and time series into predictions, we introduce a Multimodal Mixture-of-Experts (MMoE) module as a lightweight plug-in to empower Transformer-based time series models for multimodal forecasting, eliminating the need for explicit representation-level alignment. Experimentally, our proposed TiMi demonstrates consistent state-of-the-art performance on sixteen real-world multimodal forecasting benchmarks, outperforming advanced baselines while offering both strong adaptability and interpretability.

时间序列多模态大模型预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。