arXiv:2508.16059cs.AIcs.CL2025-08

让大模型在每层都精准理解时间序列,提升预测准确率。

Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting

  • 用可调向量在各层融合文本与时间序列表示
  • 7个基准上均方误差降低31.8%
  • 适合需要少样本高效预测的场景

时间序列数据广泛存在于各类应用中,时间序列预测(TSF)是基础任务。随着大语言模型(LLMs)的迅猛发展,已有多种方法尝试将LLMs用于时间序列预测。然而,现有方法对时间序列信息的整合过于浅层,通常仅在输入层接入序列表示,导致其在深层逐渐衰减,文本与时间序列表示间的适配效果不佳。本文提出多层可调嵌入融合(MSEF)框架,使LLM能在所有层直接访问时间序列模式,缓解深层信息丢失问题。具体而言,MSEF利用现成的时间序列基础模型提取语义丰富的嵌入,并通过层特定的调节向量与LLM中间文本表示进行融合。这些调节向量持续优化时序与文本模态的对齐,实现分层自适应机制,保障高效的少样本学习能力。在7个基准上的实验表明,相比基线方法,MSEF在均方误差(MSE)上平均降低31.8%。代码已开源:https://github.com/One1sAll/MSEF。

原文摘要 · Abstract (English)

Time series (TS) data are ubiquitous across various application areas, rendering time series forecasting (TSF) a fundamental task. With the astounding advances in large language models (LLMs), a variety of methods have been developed to adapt LLMs for time series forecasting. Despite unlocking the potential of LLMs in comprehending TS data, existing methods are inherently constrained by their shallow integration of TS information, wherein LLMs typically access TS representations at shallow layers, primarily at the input layer. This causes the influence of TS representations to progressively fade in deeper layers and eventually leads to ineffective adaptation between textual embeddings and TS representations. In this paper, we propose the Multi-layer Steerable Embedding Fusion (MSEF), a novel framework that enables LLMs to directly access time series patterns at all depths, thereby mitigating the progressive loss of TS information in deeper layers. Specifically, MSEF leverages off-the-shelf time series foundation models to extract semantically rich embeddings, which are fused with intermediate text representations across LLM layers via layer-specific steering vectors. These steering vectors are designed to continuously optimize the alignment between time series and textual modalities and facilitate a layer-specific adaptation mechanism that ensures efficient few-shot learning capabilities. Experimental results on seven benchmarks demonstrate significant performance improvements by MSEF compared with baselines, with an average reduction of 31.8% in terms of MSE. The code is available at https://github.com/One1sAll/MSEF.

时间序列预测大模型融合少样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。