时间序列预测中，基础模型效果不如预期，小而专的模型反而更优。
How Foundational are Foundation Models for Time Series Forecasting?
- 用预训练时间序列数据验证基础模型零样本能力受限于预训练领域。
- 微调后基础模型在真实数据上表现不优于参数更少的专用模型。
- 适合关注模型效率与任务适配性的研究者和工业应用开发者。
基础模型被设计为通用嵌入引擎，具备强大的零样本能力和微调后的优异泛化性能。尽管这在语言和视觉领域基本成立，但我们认为时间序列数据的固有多样性使其难以构建有效的基础模型。以预测为下游任务，我们证明了时间序列基础模型的零样本能力显著依赖于其预训练时覆盖的具体领域。此外，在处理未见过的真实世界时间序列数据时，微调后的基础模型虽参数量更大、内存开销更高，但其性能提升并不明显，远不及针对特定预测任务优化的小型专用模型。
原文摘要 · Abstract (English)
Foundation Models are designed to serve as versatile embedding machines, with strong zero shot capabilities and superior generalization performance when fine-tuned on diverse downstream tasks. While this is largely true for language and vision foundation models, we argue that the inherent diversity of time series data makes them less suited for building effective foundation models. We demonstrate this using forecasting as our downstream task. We show that the zero-shot capabilities of a time series foundation model are significantly influenced and tied to the specific domains it has been pretrained on. Furthermore, when applied to unseen real-world time series data, fine-tuned foundation models do not consistently yield substantially better results, relative to their increased parameter count and memory footprint, than smaller, dedicated models tailored to the specific forecasting task at hand.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。