用小波分解构建时间序列的紧凑词表,提升模型预测精度与泛化能力。
Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization
- 基于小波变换对时序数据进行频域局部化编码,生成紧凑词表
- 在42个数据集上用1024词表实现比现有模型更优的准确率和泛化性
- 擅长捕捉趋势、稀疏突变等复杂时序模式,适合实际应用
如何构建适用于时间序列预测的基础模型仍是重要开放问题。词元化是其中关键:如何为连续实值序列设计有效离散词汇?本文提出WaveToken,一种基于小波的词元化方法,使模型能直接在时频局部化空间中学习复杂表示。该方法先对输入序列进行缩放与小波分解,再对小波系数进行阈值处理与量化,最后预训练自回归模型以预测未来时段的系数。小波能同时捕捉粗粒度与细粒度结构,为时间序列提供简洁而有力的表达方式,降低学习难度。在包含42个数据集的综合基准测试中,包括域内与零样本设置,结果表明:i)WaveToken在仅使用1024个词元的情况下,预测精度优于近期提出的基础模型,并达到或超过针对各数据集专门训练的现代深度学习模型;ii)具备卓越泛化能力,在三项互补指标下平均排名最优。此外,该方法可有效捕捉趋势、稀疏突变以及频率随时间变化的非平稳序列等实际中具有挑战性的时序模式。
原文摘要 · Abstract (English)
How to best develop foundational models for time series forecasting remains an important open question. Tokenization is a crucial consideration in this effort: what is an effective discrete vocabulary for a real-valued sequential input? To address this question, we develop WaveToken, a wavelet-based tokenizer that allows models to learn complex representations directly in the space of time-localized frequencies. Our method first scales and decomposes the input time series, then thresholds and quantizes the wavelet coefficients, and finally pre-trains an autoregressive model to forecast coefficients for the forecast horizon. By decomposing coarse and fine structures in the inputs, wavelets provide an eloquent and compact language for time series forecasting that simplifies learning. Empirical results on a comprehensive benchmark, including 42 datasets for both in-domain and zero-shot settings, show that WaveToken: i) provides better accuracy than recently proposed foundation models for forecasting while using a much smaller vocabulary (1024 tokens), and performs on par or better than modern deep learning models trained specifically on each dataset; and ii) exhibits superior generalization capabilities, achieving the best average rank across all datasets for three complementary metrics. In addition, we show that our method can easily capture complex temporal patterns of practical relevance that are challenging for other recent pre-trained models, including trends, sparse spikes, and non-stationary time series with varying frequencies evolving over time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。