提出FLAM方法，让联邦学习评估结果与集中式一致。
FLAM: Evaluating Model Performance with Aggregatable Measures in Federated Learning
- 设计可聚合的评估度量，避免局部计算偏差
- 在无全局测试集时仍能等效于集中式评估
- 适合关注评估公平性的联邦学习研究者
性能评估对机器学习模型质量判断和部署决策至关重要。在联邦学习中，由于数据分散在各参与方，评估面临挑战：协调者需依赖本地计算的评估指标并进行聚合以评估全局模型。现有聚合策略（如按本地样本数加权平均）常导致结果与集中式评估不一致。当前评估定义多针对准确率，难以推广至其他指标，造成参与者评估与集中式评估之间的不一致。这违背了联邦学习目标，并导致指标计算错误。为此，我们分析了不一致的根本原因，提出FLAM方法——一种基于可聚合度量的性能评估机制，在无需全局测试数据的情况下，可实现与集中式评估等效的结果。
原文摘要 · Abstract (English)
Performance evaluation is essential for assessing the quality of machine learning (ML) models and guiding deployment decisions. In federated learning (FL), assessing the performance is challenging because data are distributed across participants. Consequently, the coordinator must rely on locally computed evaluation metrics and aggregate them to assess the global model. A key challenge is that common aggregation strategies, such as weighted averaging based on the local samples per participant, do not always produce the same results as centralized evaluation. Existing definitions of performance evaluation are largely tailored to accuracy and do not generalize to other metrics, leading to inconsistencies between participant-based and centralized evaluation. However, such discrepancies are inconsistent with the FL objective and lead to a wrong calculation of the metric. To address this issue, we examine the underlying reasons for these discrepancies and propose FLAM, a performance evaluation method based on aggregatable measures that yields the same results as centralized evaluation without the need for a global test dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。