用结构与语义双重视角,从海量图数据中精选小样本用于高效预训练。
GraphSculptor: Sculpting Pre-training Coreset for Graph Self-supervised Learning

- 基于图的结构特征和文本描述语义,双视角构建无监督核心数据集。
- 10%核心数据达到99.6%全量性能,预训练时间减少近90%。
- 理论可证明性能损失边界,适合大规模图学习场景降本增效。
图自监督学习通常依赖大规模未标注数据集,导致计算成本高昂。然而实证表明这些数据存在大量冗余——我们分析发现,均匀采样50%的图仍能保持超过96%的下游性能。为此,我们提出GraphSculptor,用于构建预训练核心数据集。不同于依赖训练时信号或仅限拓扑统计的方法,GraphSculptor提供无标签解决方案,通过两个互补视角构建核心集:内在结构与上下文语义。具体而言,结构多样性通过内在图统计量化,生成每张图的结构特征向量;语义多样性则利用预训练语言模型编码由图转文本生成的描述。GraphSculptor将两类信号融合至统一度量空间,并进行聚类感知选择,以保留联合的结构-语义多样性。我们进一步推导了核心集与全量数据预训练之间的损失差距理论界,为选择策略提供理论依据。大量实验表明,GraphSculptor能有效雕琢数据集:仅使用10%的核心集即可实现99.6%的全量性能,同时将预训练时间降低近90%,为数据高效的图预训练提供了可扩展方案。
原文摘要 · Abstract (English)
Graph self-supervised learning typically relies on large-scale unlabeled datasets, heavily inflating computational costs. However, empirical evidence suggests that these datasets contain substantial redundancy-our analysis reveals that uniformly subsampling 50% of graphs retains over 96% of downstream performance. To exploit this redundancy, we introduce GraphSculptor for pre-training coreset construction. Unlike methods dependent on additional training-time signals or limited solely to topological statistics, GraphSculptor provides a label-free solution that constructs coresets via two complementary perspectives: intrinsic structure and contextual semantics. Concretely, structural diversity is quantified using intrinsic graph statistics, yielding a structural feature vector for each graph, while semantic diversity is captured by utilizing a pre-trained language model to encode descriptions generated via graph-to-text. GraphSculptor integrates these signals into a unified metric space and performs cluster-aware selection to preserve joint structural-semantic diversity. We further derive a theoretical bound on the loss gap between coreset and full-data pre-training, offering theoretical motivation for our selection formulation. Extensive experiments demonstrate that GraphSculptor effectively sculpts the dataset: a 10% coreset achieves 99.6% of full-data performance while reducing pre-training time by nearly 90%, offering a scalable solution for data-efficient graph pre-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。