用新方法让大模型高效处理时间序列,效果超越现有技术。
Adapting Large Language Models for Time Series Modeling via a Novel Parameter-efficient Adaptation Method
- 将时间序列转为令牌嵌入,与文本提示对齐
- 动态选择适配模块,提升预测准确率
- 适合想用大模型做时间序列分析的研究者
时间序列建模在诸多实际应用中具有重要意义,但受限于数据稀疏性,预训练基础模型在该领域的进展缓慢。尽管大语言模型(LLM)在自然语言处理和计算机视觉领域表现优异,但在时间序列建模中仍面临模态对齐与推理效率难以兼顾的挑战。为此,本文提出Time-LlaMA框架:首先通过线性令牌化机制将时间序列输入转换为嵌入表示;其次实现时间序列嵌入与文本提示的对齐;最后引入动态低秩适应技术(D-LoRA),根据每条时间序列输入动态选择Transformer主干中各层最合适的LoRA模块,从而增强模型预测能力。在多个具有挑战性的真实世界时间序列任务上的实验表明,所提方法达到当前最优性能(SOTA)。
原文摘要 · Abstract (English)
Time series modeling holds significant importance in many real-world applications and has been extensively studied. While pre-trained foundation models have made impressive strides in the fields of natural language processing (NLP) and computer vision (CV), their development in time series domains has been constrained by data sparsity. A series of recent studies have demonstrated that large language models (LLMs) possess robust pattern recognition and reasoning abilities over complex sequences of tokens. However, the current literature have yet striked a high-quality balance between (a) effectively aligning the time series and natural language modalities, and (b) keeping the inference efficiency. To address the above issues, we now propose the Time-LlaMA framework. Time-LlaMA first converts the time series input into token embeddings through a linear tokenization mechanism. Second, the time series token embeddings are aligned with the text prompts. Third, to further adapt the LLM backbone for time series modeling, we have developed a dynamic low-rank adaptation technique (D-LoRA). D-LoRA dynamically chooses the most suitable LoRA modules at each layer of the Transformer backbone for each time series input, enhancing the model's predictive capabilities. Our experimental results on an extensive collection of challenging real-world time series tasks confirm that our proposed method achieves the state-of-the-art (SOTA) performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。