用语义增强让大模型更好理解时间序列,提升预测精度与可解释性。
Semantic-Enhanced Time-Series Forecasting via Large Language Models
- 引入周期性与异常特征增强语义空间,优化令牌表征
- 在多个数据集上超越当前最优方法,尤其擅长捕捉短期异常
- 轻量插件设计,冻结大模型,降低计算开销,适合工业部署
时间序列预测在金融、能源、气象和物联网等领域具有重要意义。近期研究利用大语言模型(LLM)的泛化能力进行时序预测,取得了良好效果。然而,现有方法仅关注词元级模态对齐,未弥合语言知识结构与时间序列数据模式之间的内在差异,严重限制了语义表示能力。为此,我们提出一种新型语义增强型大语言模型(SE-LLM),通过挖掘时间序列固有的周期性和异常特征,将其嵌入语义空间以增强词元表征,从而提升模型对时间序列的可解释性,激活其时序分析潜力。此外,现有基于Transformer的LLM虽擅长捕捉长程依赖,但在建模短时异常方面表现不足。因此,我们在自注意力机制中嵌入一个插件模块,有效建模长短期依赖关系,实现对时序分析的适配。本方法冻结预训练大模型并降低词元序列维度,显著减少计算消耗。实验表明,SE-LLM在多个基准数据集上均优于当前最先进方法。
原文摘要 · Abstract (English)
Time series forecasting plays a significant role in finance, energy, meteorology, and IoT applications. Recent studies have leveraged the generalization capabilities of large language models (LLMs) to adapt to time series forecasting, achieving promising performance. However, existing studies focus on token-level modal alignment, instead of bridging the intrinsic modality gap between linguistic knowledge structures and time series data patterns, greatly limiting the semantic representation. To address this issue, we propose a novel Semantic-Enhanced LLM (SE-LLM) that explores the inherent periodicity and anomalous characteristics of time series to embed into the semantic space to enhance the token embedding. This process enhances the interpretability of tokens for LLMs, thereby activating the potential of LLMs for temporal sequence analysis. Moreover, existing Transformer-based LLMs excel at capturing long-range dependencies but are weak at modeling short-term anomalies in time-series data. Hence, we propose a plugin module embedded within self-attention that models long-term and short-term dependencies to effectively adapt LLMs to time-series analysis. Our approach freezes the LLM and reduces the sequence dimensionality of tokens, greatly reducing computational consumption. Experiments demonstrate the superiority performance of our SE-LLM against the state-of-the-art (SOTA) methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。