提出双向多尺度图数据压缩框架,解决设备端训练中不同规模图数据的高效保持问题。
Bi-Directional Multi-Scale Graph Dataset Condensation via Information Bottleneck
- 双向处理大到小与小到大尺度,避免信息丢失
- 通过信息瓶颈理论确定最优中间尺度,提升压缩稳定性
- 适用于资源受限设备上的多场景图学习任务
数据压缩显著提升了模型训练效率,但在计算能力各异的设备上应用时,对不同数据规模提出了新要求。同时压缩多尺度图数据是实现设备端高效训练的核心。现有方法主要按尺度顺序(大到小或小到大)进行近似计算,但对于稀疏图的非欧几里得结构,这两种范式均存在缩放后退化和缩放前崩溃问题。其根本瓶颈在于:在生成首个子尺度图时,原始图的有效信息是否被充分保留,这直接决定所有尺度的压缩效果与一致性。本文提出一种以图神经网络为中心的新型双向多尺度图数据压缩(BiMSGC)框架,通过同时操作大到小与小到大两个方向,统一压缩范式。基于互信息理论,估计出一个最优‘中间尺度’,以最小密度图保留最大有用信息;并通过在其他尺度上优化图特征基匹配与信息瓶颈,实现稳定一致的双向压缩学习。多个数据集上的实验证明,该框架在不同尺度下的图压缩性能显著优于现有方法。
原文摘要 · Abstract (English)
Dataset condensation has significantly improved model training efficiency, but its application on devices with different computing power brings new requirements for different data sizes. Thus, condensing multiple scale graphs simultaneously is the core of achieving efficient training in different on-device scenarios. Existing efficient works for multi-scale graph dataset condensation mainly perform efficient approximate computation in scale order (large-to-small or small-to-large scales). However, for non-Euclidean structures of sparse graph data, these two commonly used paradigms for multi-scale graph dataset condensation have serious scaling down degradation and scaling up collapse problems of a graph. The main bottleneck of the above paradigms is whether the effective information of the original graph is fully preserved when consenting to the primary sub-scale (the first of multiple scales), which determines the condensation effect and consistency of all scales. In this paper, we proposed a novel GNN-centric Bi-directional Multi-Scale Graph Dataset Condensation (BiMSGC) framework, to explore unifying paradigms by operating on both large-to-small and small-to-large for multi-scale graph condensation. Based on the mutual information theory, we estimate an optimal ``meso-scale'' to obtain the minimum necessary dense graph preserving the maximum utility information of the original graph, and then we achieve stable and consistent ``bi-directional'' condensation learning by optimizing graph eigenbasis matching with information bottleneck on other scales. Encouraging empirical results on several datasets demonstrates the significant superiority of the proposed framework in graph condensation at different scales.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。