arXiv:2410.16489cs.LG2024-10被引 5

将大模型与传统时间序列模型结合,提升预测与异常检测能力

LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling

  • 用互信息模块融合大模型文本表征与传统模型时序特征
  • 在5个主流任务上达到或超越现有最佳性能
  • 动态重加权样本,优化不同损失的训练效率

时间序列建模在气象预测、异常检测等动态系统中至关重要。近期研究利用大语言模型(LLM)的强模式识别能力进行时间序列建模,但多将其作为预测主干,忽视了传统模型中的数学建模(如周期性)。为此,本文提出新框架 LLM-TS Integrator,通过互信息模块将传统时间序列模型与大模型生成的文本表示深度融合,最大化两者表征间的互信息以实现跨模态对齐。此外,针对样本在预测损失与互信息损失中重要性差异的问题,引入双权重样本重加权模块,采用两阶段优化动态调整权重。实验在五个主流时间序列任务(短/长期预测、填补、分类、异常检测)上验证了该方法的优越性,性能达到或超过当前最优水平。

原文摘要 · Abstract (English)

Time series~(TS) modeling is essential in dynamic systems like weather prediction and anomaly detection. Recent studies utilize Large Language Models (LLMs) for TS modeling, leveraging their powerful pattern recognition capabilities. These methods primarily position LLMs as the predictive backbone, often omitting the mathematical modeling within traditional TS models, such as periodicity. However, disregarding the potential of LLMs also overlooks their pattern recognition capabilities. To address this gap, we introduce \textit{LLM-TS Integrator}, a novel framework that effectively integrates the capabilities of LLMs into traditional TS modeling. Central to this integration is our \textit{mutual information} module. The core of this \textit{mutual information} module is a traditional TS model enhanced with LLM-derived insights for improved predictive abilities. This enhancement is achieved by maximizing the mutual information between traditional model's TS representations and LLM's textual representation counterparts, bridging the two modalities. Moreover, we recognize that samples vary in importance for two losses: traditional prediction and mutual information maximization. To address this variability, we introduce the \textit{sample reweighting} module to improve information utilization. This module assigns dual weights to each sample: one for prediction loss and another for mutual information loss, dynamically optimizing these weights via bi-level optimization. Our method achieves state-of-the-art or comparable performance across five mainstream TS tasks, including short-term and long-term forecasting, imputation, classification, and anomaly detection.

时间序列大模型互信息多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。