提出OLinear模型,在正交变换域实现高效多变量时间序列预测。
OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain
- 在正交变换域中解耦时序依赖,提升建模效率
- 自适应正交变换使模型在24个基准上均达最优
- 线性模块比注意力机制快一倍且性能更优
本文提出OLinear,一种基于线性结构的多变量时间序列预测模型,工作于正交变换域。传统方法多在时域直接编码解码,但序列中复杂的逐步依赖关系会限制性能。现有方法使用固定基(如傅里叶变换中的正弦余弦)进行变换,但缺乏数据适应性。我们引入OrthoTrans,一种基于正交矩阵的自适应变换,可对齐序列的时间皮尔逊相关矩阵,实现特征去相关化。该方法可作为插件模块增强现有预测器。为加强多变量表示学习,我们设计了归一化权重矩阵的线性层NormLin,其在多项任务中表现超越多头自注意机制,同时仅需约一半的浮点运算量。在24个基准、140项任务上的大量实验表明,OLinear始终达到领先性能,且作为自注意力替代品可持续提升Transformer类模型表现。代码与数据集已公开。
原文摘要 · Abstract (English)
This paper presents $\mathbf{OLinear}$, a $\mathbf{linear}$-based multivariate time series forecasting model that operates in an $\mathbf{o}$rthogonally transformed domain. Recent forecasting models typically adopt the temporal forecast (TF) paradigm, which directly encode and decode time series in the time domain. However, the entangled step-wise dependencies in series data can hinder the performance of TF. To address this, some forecasters conduct encoding and decoding in the transformed domain using fixed, dataset-independent bases (e.g., sine and cosine signals in the Fourier transform). In contrast, we utilize $\mathbf{OrthoTrans}$, a data-adaptive transformation based on an orthogonal matrix that diagonalizes the series' temporal Pearson correlation matrix. This approach enables more effective encoding and decoding in the decorrelated feature domain and can serve as a plug-in module to enhance existing forecasters. To enhance the representation learning for multivariate time series, we introduce a customized linear layer, $\mathbf{NormLin}$, which employs a normalized weight matrix to capture multivariate dependencies. Empirically, the NormLin module shows a surprising performance advantage over multi-head self-attention, while requiring nearly half the FLOPs. Extensive experiments on 24 benchmarks and 140 forecasting tasks demonstrate that OLinear consistently achieves state-of-the-art performance with high efficiency. Notably, as a plug-in replacement for self-attention, the NormLin module consistently enhances Transformer-based forecasters. The code and datasets are available at https://anonymous.4open.science/r/OLinear
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。