提出HeSRN模型,高效建模异构图语义与结构依赖。
HeSRN: Representation Learning On Heterogeneous Graphs via Slot-Aware Retentive Network
- 用槽感知结构解耦节点类型语义,缓解特征融合混淆。
- 以线性复杂度的保留机制替代自注意力,提升效率与表达力。
- 适合处理大规模异构图任务,对资源有限场景更友好。
图Transformer在捕捉长程依赖方面取得显著进展,但其二次计算复杂度及难以有效建模异构语义的问题,严重制约了其在真实异构图上的可扩展性和泛化能力。为此,我们提出HeSRN——一种新型异构槽感知保留网络,用于高效且富有表现力的异构图表示学习。HeSRN引入槽感知结构编码器,通过将异构特征投影至独立槽并利用槽归一化与基于保留的融合对齐分布,显式解耦节点类型语义,有效缓解传统Transformer模型中强制特征空间统一带来的语义纠缠。此外,我们以保留机制替代自注意力,实现线性时间复杂度下对结构与上下文依赖的建模,同时保持强表达能力。进一步采用异构保留编码器,通过多尺度保留层联合捕获局部结构信号与全局异构语义。在四个真实异构图数据集上的大量实验表明,HeSRN在节点分类任务上持续优于当前最先进异构图神经网络与图Transformer基线,在显著降低计算复杂度的同时获得更高准确率。
原文摘要 · Abstract (English)
Graph Transformers have recently achieved remarkable progress in graph representation learning by capturing long-range dependencies through self-attention. However, their quadratic computational complexity and inability to effectively model heterogeneous semantics severely limit their scalability and generalization on real-world heterogeneous graphs. To address these issues, we propose HeSRN, a novel Heterogeneous Slot-aware Retentive Network for efficient and expressive heterogeneous graph representation learning. HeSRN introduces a slot-aware structure encoder that explicitly disentangles node-type semantics by projecting heterogeneous features into independent slots and aligning their distributions through slot normalization and retention-based fusion, effectively mitigating the semantic entanglement caused by forced feature-space unification in previous Transformer-based models. Furthermore, we replace the self-attention mechanism with a retention-based encoder, which models structural and contextual dependencies in linear time complexity while maintaining strong expressive power. A heterogeneous retentive encoder is further employed to jointly capture both local structural signals and global heterogeneous semantics through multi-scale retention layers. Extensive experiments on four real-world heterogeneous graph datasets demonstrate that HeSRN consistently outperforms state-of-the-art heterogeneous graph neural networks and Graph Transformer baselines on node classification tasks, achieving superior accuracy with significantly lower computational complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。