arXiv:2411.05693cs.LG2024-11被引 1

YOSO通过压缩感知实现一次采样,大幅降低图神经网络训练开销。

YOSO: You-Only-Sample-Once via Compressed Sensing for Graph Neural Network Training

  • 仅在输入层采样一次,每轮训练后通过重建恢复节点信息
  • 训练时间平均减少75%,精度与全节点参与相当
  • 适合大规模图数据的高效训练,尤其关注低延迟场景

图神经网络(GNN)已成为分析非欧几里得数据的重要工具。训练阶段采样对降低延迟至关重要,尤其在大规模应用中。然而,随着性能需求提升,现有采样方法日趋复杂,带来显著计算开销。为此,我们提出YOSO(You-Only-Sample-Once),一种基于压缩感知(CS)的采样与重建框架:节点仅在输入层采样一次,每轮训练后在输出层进行无损重建。通过将重建过程与特定学习任务的损失函数结合,YOSO避免了传统压缩感知中如正交基计算等高成本操作,同时确保高概率下精度等同于全节点参与。在节点分类和链接预测任务上的实验表明,相比当前最优方法,YOSO平均降低75%的训练开销,且精度保持在顶尖基准水平。

原文摘要 · Abstract (English)

Graph neural networks (GNNs) have become essential tools for analyzing non-Euclidean data across various domains. During training stage, sampling plays an important role in reducing latency by limiting the number of nodes processed, particularly in large-scale applications. However, as the demand for better prediction performance grows, existing sampling algorithms become increasingly complex, leading to significant overhead. To mitigate this, we propose YOSO (You-Only-Sample-Once), an algorithm designed to achieve efficient training while preserving prediction accuracy. YOSO introduces a compressed sensing (CS)-based sampling and reconstruction framework, where nodes are sampled once at input layer, followed by a lossless reconstruction at the output layer per epoch. By integrating the reconstruction process with the loss function of specific learning tasks, YOSO not only avoids costly computations in traditional compressed sensing (CS) methods, such as orthonormal basis calculations, but also ensures high-probability accuracy retention which equivalent to full node participation. Experimental results on node classification and link prediction demonstrate the effectiveness and efficiency of YOSO, reducing GNN training by an average of 75\% compared to state-of-the-art methods, while maintaining accuracy on par with top-performing baselines.

图神经网络压缩感知采样优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。