用30亿参数小模型实现高效高精度时间序列预测。
Small but Mighty: Enhancing Time Series Forecasting with Lightweight LLMs
- 用统计特征增强提示,让数值数据与文本语义对齐。
- 30亿参数模型比70亿参数模型快3.8倍、省5.2倍内存。
- 适合需要快速部署和融合专家知识的实时预测场景。
尽管大语言模型在时间序列预测中展现出巨大潜力,但其实际应用受限于过高的计算开销和内存占用。现有方法普遍存在三大缺陷:数值时间序列模式参数利用效率低;连续时序信号与离散文本嵌入存在模态错配;难以实时融入领域专家知识。本文提出SMETimes,首次系统研究参数量小于30亿的轻量级语言模型(SLM)在高效准确的时间序列预测中的应用。核心创新包括:基于描述性统计特征的统计增强提示机制,实现数值序列与文本语义的桥梁连接;可学习参数的自适应融合嵌入架构,将时序模式与语言模型词元空间对齐;以及依托轻量模型计算效率的动态专家混合框架,动态融合基础预测与领域特定模型。在七个基准数据集上的实验表明,30亿参数的SMETimes在五个主要数据集上达到当前最优性能,训练速度比70亿参数基线快3.8倍,内存消耗降低5.2倍。显著提升学习能力,长期预测任务均方误差较传统大模型降低12.3%。消融实验证明,统计提示和跨模态融合模块分别带来15.7%和18.2%的误差下降。本工作重新定义了效率与精度的权衡边界,确立了轻量级模型在实际时间序列预测中的可行性。代码与模型已开源。
原文摘要 · Abstract (English)
While LLMs have demonstrated remarkable potential in time series forecasting, their practical deployment remains constrained by excessive computational demands and memory footprints. Existing LLM-based approaches typically suffer from three critical limitations: Inefficient parameter utilization in handling numerical time series patterns; Modality misalignment between continuous temporal signals and discrete text embeddings; and Inflexibility for real-time expert knowledge integration. We present SMETimes, the first systematic investigation of sub-3B parameter SLMs for efficient and accurate time series forecasting. Our approach centers on three key innovations: A statistically-enhanced prompting mechanism that bridges numerical time series with textual semantics through descriptive statistical features; A adaptive fusion embedding architecture that aligns temporal patterns with language model token spaces through learnable parameters; And a dynamic mixture-of-experts framework enabled by SLMs' computational efficiency, adaptively combining base predictions with domain-specific models. Extensive evaluations across seven benchmark datasets demonstrate that our 3B-parameter SLM achieves state-of-the-art performance on five primary datasets while maintaining 3.8x faster training and 5.2x lower memory consumption compared to 7B-parameter LLM baselines. Notably, the proposed model exhibits better learning capabilities, achieving 12.3% lower MSE than conventional LLM. Ablation studies validate that our statistical prompting and cross-modal fusion modules respectively contribute 15.7% and 18.2% error reduction in long-horizon forecasting tasks. By redefining the efficiency-accuracy trade-off landscape, this work establishes SLMs as viable alternatives to resource-intensive LLMs for practical time series forecasting. Code and models are available at https://github.com/xiyan1234567/SMETimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。