用SVD与Transformer结合,让图神经网络更懂边的方向性。
SVDformer: Direction-Aware Spectral Graph Embedding Learning via SVD and Transformer
- 通过SVD分解提取频谱特征,再用Transformer自适应增强关键成分。
- 在6个基准数据集上,节点分类准确率全面超越现有方法。
- 适合研究方向敏感的图学习任务,如社交网络、知识图谱。
有向图广泛用于建模真实系统中的非对称关系。然而,现有有向图神经网络常因各向同性的聚合机制和局部滤波方式,难以同时捕捉方向语义与全局结构模式。为此,本文提出SVDformer,一种融合SVD与Transformer架构的新框架,实现方向感知的图表示学习。SVDformer首先通过多头自注意力对奇异值嵌入进行精炼,自适应增强关键频谱成分并抑制高频噪声,从而实现无需显式谱核的可学习低通/高通图滤波。此外,通过将奇异向量视为方向投影基,奇异值作为缩放因子,SVDformer利用Transformer建模输入/输出边模式间的多尺度交互,显式保留边的方向性。在六个有向图基准上的大量实验表明,SVDformer在节点分类任务中持续优于当前最优的GNN及方向感知基线,确立了有向图表示学习的新范式。
原文摘要 · Abstract (English)
Directed graphs are widely used to model asymmetric relationships in real-world systems. However, existing directed graph neural networks often struggle to jointly capture directional semantics and global structural patterns due to their isotropic aggregation mechanisms and localized filtering mechanisms. To address this limitation, this paper proposes SVDformer, a novel framework that synergizes SVD and Transformer architecture for direction-aware graph representation learning. SVDformer first refines singular value embeddings through multi-head self-attention, adaptively enhancing critical spectral components while suppressing high-frequency noise. This enables learnable low-pass/high-pass graph filtering without requiring spectral kernels. Furthermore, by treating singular vectors as directional projection bases and singular values as scaling factors, SVDformer uses the Transformer to model multi-scale interactions between incoming/outgoing edge patterns through attention weights, thereby explicitly preserving edge directionality during feature propagation. Extensive experiments on six directed graph benchmarks demonstrate that SVDformer consistently outperforms state-of-the-art GNNs and direction-aware baselines on node classification tasks, establishing a new paradigm for learning representations on directed graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。