用预训练大模型提升多变量时间序列预测精度
Using Pre-trained LLMs for Multivariate Time Series Forecasting
- 将多变量时间序列映射到大模型嵌入空间，创新使用多变量分块策略
- 在真实数据集上达到与顶尖时序模型相当的预测效果
- 适合对时序预测性能有高要求的研究者和工业应用
预训练大型语言模型（LLM）蕴含大量知识且训练成本高昂。本文利用其知识迁移能力，将其应用于多变量需求时间序列预测任务。鉴于基于Transformer的方法依赖有意义的注意力信号，而不仅仅是时间序列样本，我们探索了将多变量输入时间序列映射至LLM词嵌入空间的不同方法。特别地，我们提出的多变量分块策略能有效将时序特征嵌入解码器仅的预训练Transformer中，性能媲美当前最优时序预测模型。同时，采用最新的基于权重的诊断工具验证了结果的可靠性。
原文摘要 · Abstract (English)
Pre-trained Large Language Models (LLMs) encapsulate large amounts of knowledge and take enormous amounts of compute to train. We make use of this resource, together with the observation that LLMs are able to transfer knowledge and performance from one domain or even modality to another seemingly-unrelated area, to help with multivariate demand time series forecasting. Attention in transformer-based methods requires something worth attending to -- more than just samples of a time-series. We explore different methods to map multivariate input time series into the LLM token embedding space. In particular, our novel multivariate patching strategy to embed time series features into decoder-only pre-trained Transformers produces results competitive with state-of-the-art time series forecasting models. We also use recently-developed weight-based diagnostics to validate our findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。