用文本增强时间序列预测,让大模型更懂上下文。
Context-Aware Probabilistic Modeling with LLM for Multimodal Time Series Forecasting
- 用可学习交互融合文本与时间序列,生成联合表示。
- 结合分布专家与冻结LLM,实现上下文感知的概率预测。
- 在数据少时仍稳定,适合多模态预测场景。
时间序列预测在能源市场、气候分析和交通管理等领域至关重要。然而,现有方法难以有效融合外部文本信息,并与大语言模型(LLMs)的概率特性对齐。当前方法要么通过简单提示进行浅层文本-时间序列融合,要么依赖确定性数值解码,与LLM的自回归生成范式冲突,限制了上下文感知能力和分布建模。为此,我们提出CAPTime,一种上下文感知的概率多模态时间序列预测方法,利用文本引导的抽象和自回归LLM解码。该方法首先使用预训练的时间序列编码器提取时序模式,再通过可学习交互将其与文本上下文对齐,生成联合多模态表示。通过将分布专家混合结构与冻结的LLM结合,我们在保留LLM固有分布建模能力的同时,实现了上下文感知的概率预测。在多种时间序列预测任务上的实验表明,CAPTime在多模态场景下具有更优的准确性和泛化能力。额外分析显示,其通过混合概率解码在数据稀缺场景中表现出强鲁棒性。
原文摘要 · Abstract (English)
Time series forecasting is important for applications spanning energy markets, climate analysis, and traffic management. However, existing methods struggle to effectively integrate exogenous texts and align them with the probabilistic nature of large language models (LLMs). Current approaches either employ shallow text-time series fusion via basic prompts or rely on deterministic numerical decoding that conflict with LLMs' token-generation paradigm, which limits contextual awareness and distribution modeling. To address these limitations, we propose CAPTime, a context-aware probabilistic multimodal time series forecasting method that leverages text-informed abstraction and autoregressive LLM decoding. Our method first encodes temporal patterns using a pretrained time series encoder, then aligns them with textual contexts via learnable interactions to produce joint multimodal representations. By combining a mixture of distribution experts with frozen LLMs, we enable context-aware probabilistic forecasting while preserving LLMs' inherent distribution modeling capabilities. Experiments on diverse time series forecasting tasks demonstrate the superior accuracy and generalization of CAPTime, particularly in multimodal scenarios. Additional analysis highlights its robustness in data-scarce scenarios through hybrid probabilistic decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。