arXiv:2412.00382cs.LGcs.CY2024-12被引 8

用双教师蒸馏提升图神经网络公平性,兼顾准确率

Toward Fair Graph Neural Networks Via Dual-Teacher Knowledge Distillation

  • 设计双教师蒸馏框架,分别从特征和结构学习公平知识
  • 在多个数据集上实现最优公平性且模型性能无明显下降
  • 适合关注模型公平性与实用性的图学习研究者

图神经网络(GNN)在各类实际应用中表现出色,但常因宗教、性别等敏感属性产生偏差预测,这一问题长期被忽视。现有方法多依赖部分数据训练(如仅使用节点特征或图结构),虽能提升公平性,却因信息利用不足导致模型性能下降。为此,本文提出FairDTD框架,基于双教师知识蒸馏实现公平性与模型效用的平衡。该框架引入两个面向公平性的教师模型:特征教师与结构教师,通过双路蒸馏使学生模型学习公平表示,同时充分利用完整图数据以减少性能损失。为增强知识传递,还引入图级蒸馏提供间接信息补充,并设计节点级温度模块提升公平知识的全面迁移。在多个基准数据集上的实验表明,FairDTD在保持高模型性能的同时实现了最优公平性,验证了其在图神经网络公平表征学习中的有效性。

原文摘要 · Abstract (English)

Graph Neural Networks (GNNs) have demonstrated strong performance in graph representation learning across various real-world applications. However, they often produce biased predictions caused by sensitive attributes, such as religion or gender, an issue that has been largely overlooked in existing methods. Recently, numerous studies have focused on reducing biases in GNNs. However, these approaches often rely on training with partial data (e.g., using either node features or graph structure alone), which can enhance fairness but frequently compromises model utility due to the limited utilization of available graph information. To address this tradeoff, we propose an effective strategy to balance fairness and utility in knowledge distillation. Specifically, we introduce FairDTD, a novel Fair representation learning framework built on Dual-Teacher Distillation, leveraging a causal graph model to guide and optimize the design of the distillation process. Specifically, FairDTD employs two fairness-oriented teacher models: a feature teacher and a structure teacher, to facilitate dual distillation, with the student model learning fairness knowledge from the teachers while also leveraging full data to mitigate utility loss. To enhance information transfer, we incorporate graph-level distillation to provide an indirect supplement of graph information during training, as well as a node-specific temperature module to improve the comprehensive transfer of fair knowledge. Experiments on diverse benchmark datasets demonstrate that FairDTD achieves optimal fairness while preserving high model utility, showcasing its effectiveness in fair representation learning for GNNs.

图神经网络公平性知识蒸馏因果建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。