用瓦瑟斯坦损失优化时间序列大模型，提升预测精度。
Fine-Tuning a Time Series Foundation Model with Wasserstein Loss
- 用瓦瑟斯坦损失替代交叉熵，考虑预测值间距离
- 在22个零样本数据集上点预测性能显著提升
- 适合做时间序列建模与预测的科研人员参考
受自然语言处理中大语言模型进展的启发，近年来研究者们致力于构建时间序列预测的基础模型。一种方法是使用交叉熵损失，在分词后的时间序列数据上训练大语言模型架构。尽管该方法已展现良好效果，但交叉熵损失主要用于分类任务，未考虑类别间的距离关系。为此，本文提出采用瓦瑟斯坦损失来优化此类架构。为验证有效性，我们在22个零样本时间序列数据集上对基础模型进行微调，对比了交叉熵损失与瓦瑟斯坦损失的表现。结果表明，用瓦瑟斯坦损失替换交叉熵损失可显著提升点预测性能。
原文摘要 · Abstract (English)
Inspired by recent advancements in large language models (LLMs) for Natural Language Processing (NLP), there has been a surge in research focused on developing foundational models for time series forecasting. One approach involves training LLM architectures on tokenized time series data using cross-entropy loss. Although this method has demonstrated promising results, cross-entropy loss is primarily designed for classification tasks and does not account for the distance between classes. To address this limitation, we propose using the Wasserstein loss for such architectures. To validate our approach, we fine-tuned a foundational time series model on $22$ zero-shot datasets, comparing the performance of cross-entropy loss with that of Wasserstein loss. Our results demonstrate that replacing cross-entropy loss with Wasserstein loss significantly improves point estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。