对比大模型与小模型在时间序列预测中的表现，发现预训练大模型并非总是更优。
Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?
- 对比预训练大模型与传统小模型在多个数据集上的预测表现。
- 发现预训练模型在部分任务上精度更高，但计算成本显著增加。
- 适合需要高精度且资源充足的场景，简单模型仍具竞争力。
大规模预训练模型在多个领域展现出卓越能力，但在时间序列预测中的有效性仍缺乏深入研究。本文通过实证分析，考察了在多样化数据集上训练的预训练大规模时间序列模型（LSTSMs，如Moirai、TimeGPT）是否优于传统非预训练的小规模Transformer模型。我们在多个基准测试中评估了模型的准确性、计算效率和可解释性。结果揭示了预训练大模型的优势与局限，明确了其在特定场景下的适用性，并指出在某些情况下，更简单的模型依然具备竞争力。
原文摘要 · Abstract (English)
Large pre-trained models have demonstrated remarkable capabilities across domains, but their effectiveness in time series forecasting remains understudied. This work empirically examines whether pre-trained large-scale time series models (LSTSMs) trained on diverse datasets can outperform traditional non-pretrained small-scale transformers in forecasting tasks. We analyze state-of-the-art (SOTA) pre-trained universal time series models (e.g., Moirai, TimeGPT) alongside conventional transformers, evaluating accuracy, computational efficiency, and interpretability across multiple benchmarks. Our findings reveal the strengths and limitations of pre-trained LSTSMs, providing insights into their suitability for time series tasks compared to task-specific small-scale architectures. The results highlight scenarios where pretraining offers advantages and where simpler models remain competitive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。