Sentinel通过双注意力机制同时捕捉时间与通道依赖，提升多变量时序预测精度。
Sentinel: Multi-Patch Transformer with Temporal and Channel Attention for Time Series Forecasting
- 采用多片段注意力结构，将序列分块后自然融入Transformer架构。
- 在多个标准数据集上表现优于或接近现有最优模型。
- 适合需要同时建模时间与通道关系的复杂时序预测任务。
基于Transformer的时序预测近期受到广泛关注，因其具备建模序列数据的能力。然而，当前主流架构通常只关注时间或通道间依赖，限制了其在多变量时序预测中的效果，而这两类依赖均至关重要。本文提出Sentinel，一种全Transformer架构：编码器用于从通道维度提取上下文信息，解码器则用于捕捉时间维度上的因果关系与依赖。此外，引入多片段注意力机制，利用分块处理将输入序列以自然方式整合进Transformer，替代传统的多头分裂过程。在标准基准上的大量实验表明，由于能够同时‘监控’时间和通道维度，Sentinel在性能上优于或相当于是当前最先进方法。
原文摘要 · Abstract (English)
Transformer-based time series forecasting has recently gained strong interest due to the ability of transformers to model sequential data. Most of the state-of-the-art architectures exploit either temporal or inter-channel dependencies, limiting their effectiveness in multivariate time-series forecasting where both types of dependencies are crucial. We propose Sentinel, a full transformer-based architecture composed of an encoder able to extract contextual information from the channel dimension, and a decoder designed to capture causal relations and dependencies across the temporal dimension. Additionally, we introduce a multi-patch attention mechanism, which leverages the patching process to structure the input sequence in a way that can be naturally integrated into the transformer architecture, replacing the multi-head splitting process. Extensive experiments on standard benchmarks demonstrate that Sentinel, because of its ability to "monitor" both the temporal and the inter-channel dimension, achieves better or comparable performance with respect to state-of-the-art approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。