Sensorformer通过压缩全局补丁信息，高效融合变量与时间依赖关系。
Sensorformer: Cross-patch attention with global-patch compression is effective for high-dimensional multivariate time series forecasting
- 先压缩全局补丁信息，再统一建模跨变量与跨时间依赖
- 在9个真实数据集上超越现有方法，计算复杂度降低为原模型的1/ Patch_num
- 适合处理存在动态因果时滞的高维多变量时间序列预测
现有基于Transformer的多变量时间序列预测方法中，iTransformer仅显式提取变量间依赖，PatchTST仅显式提取时间依赖，二者均显著优于同时建模两类依赖的通道相关型Transformer。这表明当前方法仍难以有效融合时间与变量间信息。我们归因于变量间因果关系存在的动态时滞。为此提出Sensorformer：先压缩全局补丁信息，再从压缩表示中同步提取跨变量与跨时间依赖。该方法能有效捕捉正确变量关联与因果关系，即使存在动态因果时滞；同时将纯跨补丁自注意力的计算复杂度从$O(D^2 \cdot Patch\_num^2 \cdot d\_model)$降至$O(D^2 \cdot Patch\_num \cdot d\_model)$。在9个主流真实世界多变量时间序列预测数据集上的广泛对比与消融实验验证了其优越性。Sensorformer代码及复现脚本已公开于https://github.com/BigYellowTiger/Sensorformer。
原文摘要 · Abstract (English)
Among the existing Transformer-based multivariate time series forecasting methods, iTransformer, which treats each variable sequence as a token and only explicitly extracts cross-variable dependencies, and PatchTST, which adopts a channel-independent strategy and only explicitly extracts cross-time dependencies, both significantly outperform most Channel-Dependent Transformer that simultaneously extract cross-time and cross-variable dependencies. This indicates that existing Transformer-based multivariate time series forecasting methods still struggle to effectively fuse these two types of information. We attribute this issue to the dynamic time lags in the causal relationships between different variables. Therefore, we propose a new multivariate time series forecasting Transformer, Sensorformer, which first compresses the global patch information and then simultaneously extracts cross-variable and cross-time dependencies from the compressed representations. Sensorformer can effectively capture the correct inter-variable correlations and causal relationships, even in the presence of dynamic causal lags between variables, while also reducing the computational complexity of pure cross-patch self-attention from $O(D^2 \cdot Patch\_num^2 \cdot d\_model)$ to $O(D^2 \cdot Patch\_num \cdot d\_model)$. Extensive comparative and ablation experiments on 9 mainstream real-world multivariate time series forecasting datasets demonstrate the superiority of Sensorformer. The implementation of Sensorformer, following the style of the Time-series-library and scripts for reproducing the main results, is publicly available at https://github.com/BigYellowTiger/Sensorformer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。