用语言模型+图感知核，仅用1%合成数据实现接近全量数据的图学习效果。
TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernel
- 将语言模型与图感知核结合，避免重复训练完整模型
- 仅用1%合成数据即达到97%全数据性能
- 适合需要高效训练文本属性图的应用场景
文本属性图（TAGs）在多个现实场景中广泛应用，其学习需同时建模文本语义与图结构。传统方法将语言模型（LM）与图神经网络（GNN）结合，但联合训练计算成本高且难扩展。数据蒸馏是降低训练开销的有前景方法，但现有方法多针对单一模态，或仍需在完整数据集上反复训练昂贵的LM-GNN模型。为此，我们提出TaLK，一种面向TAGs的有效数据蒸馏方法，通过将语言模型与图感知神经正切核耦合，实现高效蒸馏。该设计避免了在全数据集上重复联合训练，同时保留文本与结构信息，支持有效TAG学习。在多个TAG基准测试中，TaLK持续优于现有基线，在仅使用1%合成数据时，性能可达全数据集的97%。
原文摘要 · Abstract (English)
Text-attributed graphs (TAGs) are widely used in many real-world domains, and learning on TAGs requires jointly modeling text semantics and graph structure. A standard approach for modeling TAGs is to combine a language model (LM) and a graph neural network (GNN), but joint training is computationally expensive and difficult to scale. Dataset distillation is a promising way to reduce training costs, but existing methods are not well suited to TAGs because they are typically designed for a single modality or still require repeatedly training expensive LM-GNN models on the full dataset during distillation. To address this, we propose TaLK, an effective dataset distillation method for TAGs that couples an LM with a graph-aware neural tangent kernel. This design enables efficient dataset distillation, avoiding repeated joint training on the full dataset while reflecting both textual and structural information for effective TAG learning. Experiments on multiple TAG benchmarks show that TaLK consistently outperforms existing baselines and achieves up to 97% of full-dataset performance with only 1% synthetic data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。