arXiv:2411.13264cs.LG2024-11被引 7

用稀疏注意力自动识别时间序列因果关系,无需预设滞后长度。

Transformers with Sparse Attention for Granger Causality

  • 先时序后变量的双阶段注意力机制,动态选择关键历史时刻。
  • 在多个合成数据集上优于传统VAR方法,且能自适应滞后依赖。
  • 适合需要自动发现因果链的时间序列分析任务。

时序因果分析旨在理解随时间变化的可观测变量背后的潜在因果关系。基于深度学习的方法(如Transformer)正被广泛用于捕捉超出相关性的时序动态与因果关系。近期研究指出,Transformer的自注意力权重可作为因果关联的有效指示器。本文提出一种自注意力模块的新改进方法,通过两步策略建立多变量时间序列中具有异质滞后依赖的因果链接:首先执行时序注意力,再跨时间步对变量间注意力进行逐个掩码,以计算格兰杰因果指数。该方法的核心创新在于模型能自主判断并选取对预测最显著的历史时间点,无需人工设定固定滞后值。我们在多个合成基准数据集上进行了充分实验,验证了该方法的有效性,并与假设固定滞后长度的传统向量自回归格兰杰因果方法进行了对比。

原文摘要 · Abstract (English)

Temporal causal analysis means understanding the underlying causes behind observed variables over time. Deep learning based methods such as transformers are increasingly used to capture temporal dynamics and causal relationships beyond mere correlations. Recent works suggest self-attention weights of transformers as a useful indicator of causal links. We leverage this to propose a novel modification to the self-attention module to establish causal links between the variables of multivariate time-series data with varying lag dependencies. Our Sparse Attention Transformer captures causal relationships using a two-fold approach - performing temporal attention first followed by attention between the variables across the time steps masking them individually to compute Granger Causality indices. The key novelty in our approach is the ability of the model to assert importance and pick the most significant past time instances for its prediction task against manually feeding a fixed time lag value. We demonstrate the effectiveness of our approach via extensive experimentation on several synthetic benchmark datasets. Furthermore, we compare the performance of our model with the traditional Vector Autoregression based Granger Causality method that assumes fixed lag length.

因果推断时间序列Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。