用自蒸馏提升GNN预测不确定性量化效率与精度
GNN's Uncertainty Quantification using Self-Distillation
- 同一网络充当师生模型，避免多模型训练开销
- 在MIMIC-IV和Enzymes数据集上达到与蒙特卡洛丢弃相当效果
- 新度量能捕捉模型多样性，适合临床可信决策场景
图神经网络（GNN）在医疗领域表现优异，但其预测不确定性量化仍具挑战，而这一能力对临床信任至关重要。传统贝叶斯和集成方法虽可量化不确定性，但计算成本高；且集成方法的分歧度量无法有效反映模型多样性。本文提出一种基于自蒸馏的新方法，通过同一网络同时作为教师与学生模型，无需独立训练多个网络。为增强自蒸馏效果，设计了一种加权不确定性度量，能捕捉网络分类器间的差异性。在MIMIC-IV和Enzymes两个图数据集上评估显示，该方法在保持性能的同时，能有效捕获模型预测不确定性，精度接近蒙特卡洛丢弃和集成方法。代码已开源。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have shown remarkable performance in the healthcare domain. However, what remained challenging is quantifying the predictive uncertainty of GNNs, which is an important aspect of trustworthiness in clinical settings. While Bayesian and ensemble methods can be used to quantify uncertainty, they are computationally expensive. Additionally, the disagreement metric used by ensemble methods to compute uncertainty cannot capture the diversity of models in an ensemble network. In this paper, we propose a novel method, based on knowledge distillation, to quantify GNNs' uncertainty more efficiently and with higher precision. We apply self-distillation, where the same network serves as both the teacher and student models, thereby avoiding the need to train several networks independently. To ensure the impact of self-distillation, we develop an uncertainty metric that captures the diverse nature of the network by assigning different weights to each GNN classifier. We experimentally evaluate the precision, performance, and ability of our approach in distinguishing out-of-distribution data on two graph datasets: MIMIC-IV and Enzymes. The evaluation results demonstrate that the proposed method can effectively capture the predictive uncertainty of the model while having performance similar to that of the MC Dropout and ensemble methods. The code is publicly available at https://github.com/tailabTMU/UQ_GNN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。