提出多维度评估联邦学习中客户端贡献的新方法
Beyond performance-wise Contribution Evaluation in Federated Learning
- 用谢泼德值近似法量化客户端对模型可靠、鲁棒、公平的贡献
- 发现各维度表现无统一优劣，现有单一指标无法全面评估
- 适合关注隐私保护与公平性的联邦学习研究者参考
联邦学习提供了一种注重隐私的协作学习框架，但其成功依赖于参与方的贡献。现有客户端评估方法主要关注模型性能（如准确率或损失），但这仅反映了模型整体效用的一个维度。本文探讨了被忽视的关键问题：客户端对模型可信度的贡献，具体包括可靠性（抗噪声数据能力）、鲁棒性（抵抗对抗样本）和公平性（通过人口均等性衡量）。为量化这些多维度贡献，我们采用最先进的谢泼德值近似方法，一种原理严谨的贡献归因技术。结果表明，没有一个客户端在所有维度上都表现最优，且各维度间基本独立，揭示了当前评估体系的根本缺陷：单一指标不足以实现全面评估与公平奖励分配。
原文摘要 · Abstract (English)
Federated learning offers a privacy-friendly collaborative learning framework, yet its success, like any joint venture, hinges on the contributions of its participants. Existing client evaluation methods predominantly focus on model performance, such as accuracy or loss, which represents only one dimension of a machine learning model's overall utility. In contrast, this work investigates the critical, yet overlooked, issue of client contributions towards a model's trustworthiness -- specifically, its reliability (tolerance to noisy data), resilience (resistance to adversarial examples), and fairness (measured via demographic parity). To quantify these multifaceted contributions, we employ the state-of-the-art approximation of the Shapley value, a principled method for value attribution. Our results reveal that no single client excels across all dimensions, which are largely independent from each other, highlighting a critical flaw in current evaluation scheme: no single metric is adequate for comprehensive evaluation and equitable rewarding allocation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。