解决联邦学习中客户端预测不靠谱的问题，让模型更可信。
Who to Trust? Aggregating Client Predictions in Federated Distillation
- 用密度不确定度评估客户端预测可靠性，动态加权
- 高数据异构下准确率提升，低异构时保持稳定
- 适合数据分布差异大的联邦学习场景
在数据异构（如类别不匹配）情况下，客户端对陌生类别的预测可能不可靠。简单平均这些预测会污染用于知识蒸馏的教师信号。本文对联邦蒸馏进行理论分析，表明在共享公共数据集上聚合客户端预测可收敛至最优邻域，邻域大小由聚合质量决定。进一步提出两种不确定性感知聚合方法：UWA 和 sUWA，利用基于密度的不确定性估计来降低不可靠预测的权重。图像与文本分类基准上的实验表明，当数据异构性高时，本方法表现显著优于标准平均；在异构性低时则性能相当。
原文摘要 · Abstract (English)
Under data heterogeneity (e.g., $\textit{class mismatch}$), clients may produce unreliable predictions for instances belonging to unfamiliar classes. An equally weighted combination of such predictions can corrupt the teacher signal used for distillation. In this paper, we provide a theoretical analysis of Federated Distillation and show that aggregating client predictions on a shared public dataset converges to a neighborhood of the optimum, where the neighborhood size is governed by the aggregation quality. We further propose two uncertainty-aware aggregation methods, $\mathbf{UWA}$ and $\mathbf{sUWA}$, which leverage density-based uncertainty estimates to down-weight unreliable client predictions. Experiments on image and text classification benchmarks demonstrate that our methods are particularly effective under high data heterogeneity, while matching standard averaging when heterogeneity is low.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。