用线性专家混合与文本融合,提升时序预测精度与效率。
LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting
- 采用多历史长度线性专家混合,结合文本特征自适应融合。
- 相比现有模型,预测误差更低,计算开销更小。
- 适合需要高效高精度时序预测的工业场景。
近期研究显示,大型语言模型(LLMs)凭借其强大的自然语言理解能力,可有效用于真实世界的时间序列预测。然而,将时间序列映射到LLM的语义空间会带来高昂的计算成本和推理复杂度,尤其在长序列生成时更为显著。本文基于线性模型在时序领域的进展,提出一种增强型线性专家混合模型(LeMoLE),实现精确且高效的时序预测。该方法引入多个回溯长度的线性专家,并设计一种新型多模态融合机制,根据预训练大模型提取的文本特征动态组合专家。通过重新审视现有时间序列大模型对时序与语言空间对齐的必要性,进一步探讨其在时序预测中的效率与有效性。实验表明,所提出的LeMoLE模型在预测误差上优于现有LLM模型,同时具有更高的计算效率。
原文摘要 · Abstract (English)
Recent research has shown that large language models (LLMs) can be effectively used for real-world time series forecasting due to their strong natural language understanding capabilities. However, aligning time series into semantic spaces of LLMs comes with high computational costs and inference complexity, particularly for long-range time series generation. Building on recent advancements in using linear models for time series, this paper introduces an LLM-enhanced mixture of linear experts for precise and efficient time series forecasting. This approach involves developing a mixture of linear experts with multiple lookback lengths and a new multimodal fusion mechanism. The use of a mixture of linear experts is efficient due to its simplicity, while the multimodal fusion mechanism adaptively combines multiple linear experts based on the learned features of the text modality from pre-trained large language models. In experiments, we rethink the need to align time series to LLMs by existing time-series large language models and further discuss their efficiency and effectiveness in time series forecasting. Our experimental results show that the proposed LeMoLE model presents lower prediction errors and higher computational efficiency than existing LLM models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。