arXiv:2510.19710cs.LG2025-10NeurIPS被引 6

轻量级时序预测模型,小数据训练也能精准泛化。

SEMPO: Lightweight Foundation Models for Time Series Forecasting

  • 用能量感知频谱分解捕捉高低频信号,提升预训练效率。
  • 通过提示路由机制实现参数高效适配,支持多数据集泛化。
  • 仅需小规模数据预训练,适合边缘设备部署。

近期大规模预训练模型在时序预测领域取得显著进展,但现有时序基础模型通常具有庞大网络结构,需在大规模数据上进行预训练,难以部署于资源受限环境。针对这一性能与成本的矛盾,我们提出SEMPO,一种轻量级基础模型,可在较小规模数据上预训练,仍具备强泛化能力。SEMPO包含两个核心模块:1)能量感知频谱分解模块,通过建模高能与低能但信息丰富的频率信号,显著提升预训练数据利用率;2)基于提示混合的Transformer,通过小规模数据特定提示学习异构时序模式,并自适应地将时间序列标记路由至提示驱动专家,实现跨数据集与领域的参数高效适配。实验在涵盖16个数据集的两个大规模基准上验证,SEMPO在零样本和少样本预测场景下均优于现有方法。代码与数据已公开于https://github.com/mala-lab/SEMPO。

原文摘要 · Abstract (English)

The recent boom of large pre-trained models witnesses remarkable success in developing foundation models (FMs) for time series forecasting. Despite impressive performance across diverse downstream forecasting tasks, existing time series FMs possess massive network architectures and require substantial pre-training on large-scale datasets, which significantly hinders their deployment in resource-constrained environments. In response to this growing tension between versatility and affordability, we propose SEMPO, a novel lightweight foundation model that requires pretraining on relatively small-scale data, yet exhibits strong general time series forecasting. Concretely, SEMPO comprises two key modules: 1) energy-aware SpEctral decomposition module, that substantially improves the utilization of pre-training data by modeling not only the high-energy frequency signals but also the low-energy yet informative frequency signals that are ignored in current methods; and 2) Mixture-of-PrOmpts enabled Transformer, that learns heterogeneous temporal patterns through small dataset-specific prompts and adaptively routes time series tokens to prompt-based experts for parameter-efficient model adaptation across different datasets and domains. Equipped with these modules, SEMPO significantly reduces both pre-training data scale and model size, while achieving strong generalization. Extensive experiments on two large-scale benchmarks covering 16 datasets demonstrate the superior performance of SEMPO in both zero-shot and few-shot forecasting scenarios compared with state-of-the-art methods. Code and data are available at https://github.com/mala-lab/SEMPO.

时序预测轻量模型提示学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。