用大模型的‘物理规律’指导小模型,实现低成本高精度时间序列预测。
SVTime: Small Time Series Forecasting Models Informed by "Physics" of Large Vision Model Forecasters
- 模仿大视觉模型的归纳偏置,设计带约束的线性层小模型
- 在8个数据集上超越主流轻量模型,参数量少1000倍仍媲美大模型
- 适合资源有限的用户,如中小企业做高效预测
时间序列AI对动态网页内容分析至关重要,推动了具备强知识编码与跨任务迁移能力的大规模预训练模型发展。然而,这些大模型训练和推理能耗高、硬件要求严苛,全场景通用存在碳足迹与可持续性问题。针对特定任务,更紧凑、专精且高性能的小模型更具实用性,尤其适合资源受限的用户。本文提出SVTime,一种受大视觉模型(LVM)预测器启发的小型时间序列预测模型,用于长期时间序列预测(LTSF)。我们识别出大模型在LTSF中的关键归纳偏置——类似其行为的“物理规律”,并通过精心设计的线性层与约束函数将其编码进小模型。在8个基准数据集上,对比21种基线模型(涵盖轻量、复杂及预训练大模型),SVTime显著优于现有先进轻量模型,性能接近大模型,但参数量仅为大模型的10^3分之一,且支持低资源环境下的高效训练与推理。
原文摘要 · Abstract (English)
Time series AI is crucial for analyzing dynamic web content, driving a surge of pre-trained large models known for their strong knowledge encoding and transfer capabilities across diverse tasks. However, given their energy-intensive training, inference, and hardware demands, using large models as a one-fits-all solution raises serious concerns about carbon footprint and sustainability. For a specific task, a compact yet specialized, high-performing model may be more practical and affordable, especially for resource-constrained users such as small businesses. This motivates the question: Can we build cost-effective lightweight models with large-model-like performance on core tasks such as forecasting? This paper addresses this question by introducing SVTime, a novel Small model inspired by large Vision model (LVM) forecasters for long-term Time series forecasting (LTSF). Recently, LVMs have been shown as powerful tools for LTSF. We identify a set of key inductive biases of LVM forecasters -- analogous to the "physics" governing their behaviors in LTSF -- and design small models that encode these biases through meticulously crafted linear layers and constraint functions. Across 21 baselines spanning lightweight, complex, and pre-trained large models on 8 benchmark datasets, SVTime outperforms state-of-the-art (SOTA) lightweight models and rivals large models with 10^3 fewer parameters than LVMs, while enabling efficient training and inference in low-resource settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。