用离散令牌建模多变量时间序列,提升长周期预测精度与速度。
HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting
- 将时间序列转为离散令牌,通过分层架构捕捉长期趋势。
- 在5个数据集上实现更长预测长度和更高精度,优于现有生成模型。
- 适合需要高维、长周期预测的工业时序场景。
生成模型在多变量时间序列(MTS)预测中备受关注,因其能生成高保真样本。准确预测多变量时间序列的概率分布是极具挑战性但实用的任务。尽管已有研究尝试解决该问题,仍存在两大挑战:1)现有生成方法在高维多变量时间序列预测中表现不佳,难以扩展至更高维度;2)固有的高维属性限制了生成模型的预测长度。本文指出,离散令牌表示可高效建模高维MTS,且通过预测目标自身的长期趋势可显著延长预测长度。为此,我们提出一种向量量化框架——分层离散变压器(HDT),采用增强型L2归一化向量量化策略将时间序列转化为离散令牌表示,并将预测任务转化为离散令牌生成。为克服生成模型在长周期预测中的局限,提出分层离散变压器:低层捕捉目标的离散长期趋势,高层以该趋势为条件生成目标自身的离散表示,引入自身特征以拓展高维MTS的预测长度。在五个主流MTS数据集上的大量实验验证了所提方法的有效性。
原文摘要 · Abstract (English)
Generative models have gained significant attention in multivariate time series forecasting (MTS), particularly due to their ability to generate high-fidelity samples. Forecasting the probability distribution of multivariate time series is a challenging yet practical task. Although some recent attempts have been made to handle this task, two major challenges persist: 1) some existing generative methods underperform in high-dimensional multivariate time series forecasting, which is hard to scale to higher dimensions; 2) the inherent high-dimensional multivariate attributes constrain the forecasting lengths of existing generative models. In this paper, we point out that discrete token representations can model high-dimensional MTS with faster inference time, and forecasting the target with long-term trends of itself can extend the forecasting length with high accuracy. Motivated by this, we propose a vector quantized framework called Hierarchical Discrete Transformer (HDT) that models time series into discrete token representations with l2 normalization enhanced vector quantized strategy, in which we transform the MTS forecasting into discrete tokens generation. To address the limitations of generative models in long-term forecasting, we propose a hierarchical discrete Transformer. This model captures the discrete long-term trend of the target at the low level and leverages this trend as a condition to generate the discrete representation of the target at the high level that introduces the features of the target itself to extend the forecasting length in high-dimensional MTS. Extensive experiments on five popular MTS datasets verify the effectiveness of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。