通过可微架构搜索优化图注意力机制,实现性能与可解释性的双赢。
DARTS-GT: Differentiable Architecture Search for Graph Transformers with Quantifiable Instance-Specific Interpretability Analysis
- 采用异构设计分离结构编码与特征表示,用DARTS逐层筛选最优GNN算子。
- 在8个基准上4个达领先水平,且发现的架构具数据特异性模式。
- 首次量化分析注意力头与节点贡献,揭示可视化与实际重要性不一致。
图变压器(GTs)虽在图结构数据上表现强劲,但受限于固定设计和缺乏可量化的可解释性。现有先进模型在各层使用统一的图神经网络(GNN)类型,错失深度特异性组件选择优势,且复杂架构导致性能提升难以区分真实模式与虚假相关。本文通过引入不对称设计,将查询基于节点特征,键和值来自GNN变换,重构图注意力机制。在此框架下,采用可微架构搜索(DARTS)在每层动态选择最优GNN算子,实现注意力内部的深度异质性(DARTS-GT)。为理解所发现架构,提出首个针对图变压器的定量可解释性分析框架,通过头偏离度(Head-deviation)、专业化(Specialization)和聚焦度(Focus)等指标,识别驱动预测的关键注意力头与节点,支持模型对比。在八个基准上的实验表明,DARTS-GT在四个数据集上达到最先进性能,其余保持竞争力;分析揭示注意力显著性与因果重要性并不总相关,说明广泛使用的可视化方法可能遗漏真正关键组件。关键的是,由DARTS-GT发现的异质架构始终比基线更具可解释性,证明图变压器无需在性能与可解释性间取舍。
原文摘要 · Abstract (English)
Graph Transformers (GTs) have emerged as powerful architectures for graph-structured data, yet remain constrained by rigid designs and lack quantifiable interpretability. Current state-of-the-art GTs commit to fixed GNN types across all layers, missing potential benefits of depth-specific component selection, while their complex architectures become opaque where performance gains cannot be distinguished between meaningful patterns and spurious correlations. We redesign GT attention through asymmetry, decoupling structural encoding from feature representation: queries derive from node features while keys and values come from GNN transformations. Within this framework, we use Differentiable ARchiTecture Search (DARTS) to select optimal GNN operators at each layer, enabling depth-wise heterogeneity inside transformer attention itself (DARTS-GT). To understand discovered architectures, we develop the first quantitative interpretability framework for GTs through causal ablation. Our metrics (Head-deviation, Specialization, and Focus), identify which heads and nodes drive predictions while enabling model comparison. Experiments across eight benchmarks show DARTS-GT achieves state-of-the-art on four datasets while remaining competitive on others, with discovered architectures revealing dataset-specific patterns. Our interpretability analysis reveals that visual attention salience and causal importance do not always correlate, indicating widely used visualization approaches may miss components that actually matter. Crucially, heterogeneous architectures found by DARTS-GT consistently produced more interpretable models than baselines, establishing that Graph Transformers need not choose between performance and interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。