通过熵驱动方法挖掘有向图中的数据知识,提升模型性能。
Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
- 基于拓扑与节点特征的层级知识树,量化信息流
- 在14个数据集上实现当前最优性能
- 可通用适配各类图神经网络,适合图学习研究者
有向图在建模复杂拓扑系统方面具有更强表达能力,但现有图神经网络未能充分挖掘其隐含的数据知识。本文提出熵驱动的知识蒸馏框架EDEN,从数据中心视角建立层级知识树(HKT),通过节点特征与标签间的互信息优化知识传递。该方法可在14个(有向/无向)图数据集上广泛应用,涵盖同质与异质场景,支持4类下游任务。实验表明,EDEN显著提升主流图神经网络性能,达到当前最优水平,并具备良好的通用性与即插即用特性。
原文摘要 · Abstract (English)
The directed graph (digraph), as a generalization of undirected graphs, exhibits superior representation capability in modeling complex topology systems and has garnered considerable attention in recent years. Despite the notable efforts made by existing DiGraph Neural Networks (DiGNNs) to leverage directed edges, they still fail to comprehensively delve into the abundant data knowledge concealed in the digraphs. This data-level limitation results in model-level sub-optimal predictive performance and underscores the necessity of further exploring the potential correlations between the directed edges (topology) and node profiles (feature and labels) from a data-centric perspective, thereby empowering model-centric neural networks with stronger encoding capabilities. In this paper, we propose \textbf{E}ntropy-driven \textbf{D}igraph knowl\textbf{E}dge distillatio\textbf{N} (EDEN), which can serve as a data-centric digraph learning paradigm or a model-agnostic hot-and-plug data-centric Knowledge Distillation (KD) module. The core idea is to achieve data-centric ML, guided by our proposed hierarchical encoding theory for structured data. Specifically, EDEN first utilizes directed structural measurements from a topology perspective to construct a coarse-grained Hierarchical Knowledge Tree (HKT). Subsequently, EDEN quantifies the mutual information of node profiles to refine knowledge flow in the HKT, enabling data-centric KD supervision within model training. As a general framework, EDEN can also naturally extend to undirected scenarios and demonstrate satisfactory performance. In our experiments, EDEN has been widely evaluated on 14 (di)graph datasets (homophily and heterophily) and across 4 downstream tasks. The results demonstrate that EDEN attains SOTA performance and exhibits strong improvement for prevalent (Di)GNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。