arXiv:2505.23719cs.LG2025-05NeurIPS被引 103

用改进的LSTM实现跨长短时距的零样本时间序列预测

TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning

  • 采用增强版xLSTM结构,兼具状态追踪与上下文学习能力
  • 在GiftEval和Chronos-ZS上超越多个大模型,长短期预测均领先
  • 适合缺乏训练数据、需快速部署的场景,如非专业用户应用

上下文学习使大语言模型仅通过提示中的示例即可完成任务,近年来被用于时间序列预测,实现零样本预测,让非专家也能使用强大工具,并在数据稀缺时提升性能。现有方法多依赖Transformer架构,尽管在语言任务中表现优异,但在时间序列预测中常不如LSTM等循环模型。而传统LSTM虽擅长建模时间序列,却缺乏强上下文学习能力。本文提出TiRex,基于具备优秀上下文学习能力的xLSTM,保留状态追踪特性,特别适合长期预测。为强化其状态追踪能力,引入训练阶段掩码策略CPM。TiRex在HuggingFace基准GiftEval和Chronos-ZS上达到新最佳性能,显著优于更大模型,包括TabPFN-TS(Prior Labs)、Chronos Bolt(Amazon)、TimesFM(Google)和Moirai(Salesforce),在短、长期预测中均表现突出。

原文摘要 · Abstract (English)

In-context learning, the ability of large language models to perform tasks using only examples provided in the prompt, has recently been adapted for time series forecasting. This paradigm enables zero-shot prediction, where past values serve as context for forecasting future values, making powerful forecasting tools accessible to non-experts and increasing the performance when training data are scarce. Most existing zero-shot forecasting approaches rely on transformer architectures, which, despite their success in language, often fall short of expectations in time series forecasting, where recurrent models like LSTMs frequently have the edge. Conversely, while LSTMs are well-suited for time series modeling due to their state-tracking capabilities, they lack strong in-context learning abilities. We introduce TiRex that closes this gap by leveraging xLSTM, an enhanced LSTM with competitive in-context learning skills. Unlike transformers, state-space models, or parallelizable RNNs such as RWKV, TiRex retains state-tracking, a critical property for long-horizon forecasting. To further facilitate its state-tracking ability, we propose a training-time masking strategy called CPM. TiRex sets a new state of the art in zero-shot time series forecasting on the HuggingFace benchmarks GiftEval and Chronos-ZS, outperforming significantly larger models including TabPFN-TS (Prior Labs), Chronos Bolt (Amazon), TimesFM (Google), and Moirai (Salesforce) across both short- and long-term forecasts.

时间序列零样本LSTM预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。