arXiv:2505.18088cs.LG2025-05被引 2

让图神经网络在简单任务时提前退出,提升效率且不损失精度。

Early-Exit Graph Neural Networks

  • 基于对称性设计稳定中间表示,支持安全提前退出。
  • 可训练的置信度头实现节点或图级别的动态终止,适应不同难度输入。
  • 在异质图和长程依赖任务中平衡准确率与计算效率,适合资源受限场景。

早期退出机制使深度神经网络在预测置信度高时提前终止推理,从而降低简单输入的延迟和能耗,同时在困难样本上保持全深度精度。类似地,为图神经网络(GNN)引入早期退出机制,可在简单图上动态减少计算深度以提升效率,同时在复杂图上保留全深度以捕捉细微关系。然而,在存在过平滑、过压缩或梯度消失等问题的深层GNN中,该机制尚未被充分探索。为此,我们提出对称-反对称图神经网络(SAS-GNN),其基于对称性的归纳偏置生成稳定的中间表示,支持安全的早期退出。在此基础上,我们构建了早期退出图神经网络(EEGNN),通过可端到端训练的置信度感知退出头,在节点或图层面实现即时终止。实验表明,EEGNN能学习任务驱动的退出策略,在异质图和长程任务中表现优异。即使未超越最强基线,仍持续提供有利的准确率-效率权衡,得益于其自适应且参数高效的设计。代码将公开以复现实验。

原文摘要 · Abstract (English)

Early-exit mechanisms allow deep neural networks to stop inference once prediction confidence is high, reducing latency and energy on easy inputs while retaining full-depth accuracy on harder ones. Similarly, adding early exit mechanisms to Graph Neural Networks (GNNs), the go-to models for graph-structured data, allows for dynamic trading depth for confidence on simple graphs while maintaining full-depth accuracy on harder ones to capture intricate relationships. Yet, their potential in deep GNNs, where over-smoothing, over-squashing or more generally vanishing gradients prevent these model to properly learn, remains largely unexplored. To address this, we introduce Symmetric-Anti-Symmetric GNNs (SAS-GNN), whose symmetry-based inductive biases yield stable intermediate representations that support safe early exits. Building on this backbone, we propose Early-Exit GNNs (EEGNNs), which attach confidence-aware exit neural heads which are trainable end-to-end based on the task objective, enabling on-the-fly termination at node or graph level. Experiments show that EEGNNs learn task-driven exit strategies, while achieving competitive results on heterophilic graphs and long-range tasks. Even when not outperforming the strongest baselines, EEGNNs consistently deliver favorable accuracy-efficiency trade-offs thanks to their adaptive and parameter-efficient design. We plan to release the code to reproduce our experiments.

图神经网络早期退出效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。