将Transformer重构为图卷积网络,提升时序建模的可解释性。
Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series Modeling
- 用动态邻接矩阵解释注意力机制,类比图卷积运算。
- 在多个基准上性能相当,但预测过程更透明可解释。
- 适合关注模型可解释性的时序分析研究者。
Transformer在时序建模中表现优异,但其内部机制仍不清晰。本文揭示了Transformer编码器与图卷积网络(GCN)的本质等价性:前向传播中,注意力分布矩阵充当动态邻接矩阵,其与后续变换共同实现类似图卷积的计算;反向传播中,值投影和前馈层的更新动态与GCN参数更新一致。基于此统一理论视角,我们提出Fighter(灵活图卷积变压器),简化架构,移除冗余线性投影,并引入多跳图聚合。该视角使不同尺度的时间依赖关系以图边形式显式表达。在标准预测基准上的实验表明,Fighter达到竞争性性能,同时提供更清晰的机制可解释性。
原文摘要 · Abstract (English)
Transformers have achieved remarkable success in time series modeling, yet their internal mechanisms remain opaque. This work demystifies the Transformer encoder by establishing its fundamental equivalence to a Graph Convolutional Network (GCN). We show that in the forward pass, the attention distribution matrix serves as a dynamic adjacency matrix, and its composition with subsequent transformations performs computations analogous to graph convolution. Moreover, we demonstrate that in the backward pass, the update dynamics of value and feed-forward projections mirror those of GCN parameters. Building on this unified theoretical reinterpretation, we propose \textbf{Fighter} (Flexible Graph Convolutional Transformer), a streamlined architecture that removes redundant linear projections and incorporates multi-hop graph aggregation. This perspective yields an explicit and interpretable representation of temporal dependencies across different scales, naturally expressed as graph edges. Experiments on standard forecasting benchmarks confirm that Fighter achieves competitive performance while providing clearer mechanistic interpretability of its predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。