对比30个数据集,找出使用大模型预测时间序列的最优时机。
When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters

- 通过零样本与微调对比,系统分析大模型在不同数据量下的表现
- 在15个数据集上大模型始终优于传统方法,24至8361样本达盈亏平衡点
- 提出可快速决策的两步框架,适合没资源试错的从业者
部署时间序列基础模型需投入GPU资源与工程成本,且未必优于XGBoost。本文首次系统开展盈亏平衡分析,评估30个基准数据集上零样本与LoRA微调的基础模型(Chronos、Moirai、Lag-Llama)在2%至100%训练集比例下对经典基线(Naive、ETS、ARIMA、XGBoost)的表现。在15个数据集中,基础模型在所有训练比例下均更优,部署无需数据量门槛;在6个数据集上,仅需2%数据(21–2,768样本)即被传统方法超越;其余9个数据集的盈亏平衡点介于24到8,361样本之间。一个普适规则:若训练样本少于700且存在明显周期性,直接使用零样本大模型,跳过微调——此规则可立即解决10个部署问题。反常规的是,LoRA微调可能在短序列上导致性能下降。研究提出两步决策框架:先判断序列长度与季节性强度,仅在必要时进行5%-10%小规模预实验,帮助从业者在投入完整基础设施前做出选择。四个数据集特征提示潜在机制假设,但当前规模下实现可靠自动化预测仍为开放问题。代码、基准与决策工具已开源。
原文摘要 · Abstract (English)
Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We provide the first systematic break-even analysis answering when this investment pays off. Across 30 benchmark datasets, we compare zero-shot and LoRA fine-tuned foundation models (Chronos, Moirai, Lag-Llama) against classical baselines (Naive, ETS, ARIMA, XGBoost) at six training set sizes from 2% to 100% of available data. Foundation models outperform classical methods at every evaluated training fraction on 15 of 30 datasets -- GPU deployment is unconditionally justified on these regardless of data volume. On 6 datasets, classical methods surpass zero-shot foundation models with as little as 2% of training data (21-2,768 samples); on the remaining 9, break-even ranges from 24 to 8,361 samples. One robust deployment rule requires no model training: if n_train < 700 and seasonality is non-negligible, use FM zero-shot and skip fine-tuning -- this resolves 10 of 30 deployment decisions immediately. Contrary to common practice, LoRA fine-tuning can actively degrade performance on short series. We operationalise these findings as a two-step decision framework -- compute dataset length and seasonality strength, run a brief 5-10% pilot only if needed -- enabling practitioners to make the FM-versus-classical decision before committing to full infrastructure. Four dataset features motivate mechanistic hypotheses for the remaining cases, though reliable automated prediction at this benchmark scale remains an open problem. Code, benchmark, and decision tools are available at https://github.com/nicolaisi/fm-breakeven.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。