用语言模型融合时序与文本信息,提升预测精度
CC-Time: Cross-Model and Cross-Modality Time Series Forecasting
- 结合时序数据和文本描述,跨模态建模时间依赖
- 在9个真实数据集上达到最优性能,少样本下也有效
- 适合需要多源信息融合的时序预测场景
随着预训练语言模型(PLMs)在自然语言处理以外领域的成功,其在时序预测(TSF)中的应用受到关注并展现出巨大潜力。然而,现有基于PLM的时序预测方法仍未能充分发挥语言模型强大的序列建模能力,预测精度不足。为此,本文提出一种基于语言模型的跨模型与跨模态时序预测方法(CC-Time)。从两个角度探索PLM在时序预测中的潜力:1)哪些时序特征可由PLM建模;2)仅依赖PLM是否足够构建时序模型。首先,CC-Time引入跨模态学习,利用时序序列及其对应文本描述,在语言模型中联合建模时间依赖性和通道相关性。其次,提出跨模型融合模块,自适应地融合PLM与专用时序模型的知识,形成对时序模式更全面的建模。在九个真实世界数据集上的大量实验表明,CC-Time在全数据训练和少样本学习场景下均取得当前最优预测精度。
原文摘要 · Abstract (English)
With the success of pre-trained language models (PLMs) in various application fields beyond natural language processing, language models have raised emerging attention in the field of time series forecasting (TSF) and have shown great prospects. However, current PLM-based TSF methods still fail to achieve satisfactory prediction accuracy matching the strong sequential modeling power of language models. To address this issue, we propose Cross-Model and Cross-Modality Learning with PLMs for time series forecasting (CC-Time). We explore the potential of PLMs for time series forecasting from two aspects: 1) what time series features could be modeled by PLMs, and 2) whether relying solely on PLMs is sufficient for building time series models. In the first aspect, CC-Time incorporates cross-modality learning to model temporal dependency and channel correlations in the language model from both time series sequences and their corresponding text descriptions. In the second aspect, CC-Time further proposes the cross-model fusion block to adaptively integrate knowledge from the PLMs and time series model to form a more comprehensive modeling of time series patterns. Extensive experiments on nine real-world datasets demonstrate that CC-Time achieves state-of-the-art prediction accuracy in both full-data training and few-shot learning situations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。