提出首个统一基准，评估联邦学习中数据异构性测量方法的优劣。
Benchmarking Data Heterogeneity Evaluation Approaches for Personalized Federated Learning
- 构建六种主流方法的统一评测框架，支持公平对比。
- 在五种标准非独立同分布设置下验证性能差异。
- 帮助选择合适评估方法，提升个性化联邦学习效果。
当前对客户端本地数据集统计异构性的测量方法研究分散，缺乏统一基准进行公平比较。本文提出首个系统性评测框架，涵盖六种代表性方法，在五种标准非独立同分布联邦学习设置下开展全面实验，揭示不同方法在各类场景下的优劣表现。该框架为个性化联邦学习方案设计、特定应用场景下异构性评估方法的选择以及协作训练中的公平性问题提供了有力指导。代码已开源：https://github.com/Xiaoni-61/DH-Benchmark。
原文摘要 · Abstract (English)
There is growing research interest in measuring the statistical heterogeneity of clients' local datasets. Such measurements are used to estimate the suitability for collaborative training of personalized federated learning (PFL) models. Currently, these research endeavors are taking place in silos and there is a lack of a unified benchmark to provide a fair and convenient comparison among various approaches in common settings. We aim to bridge this important gap in this paper. The proposed benchmarking framework currently includes six representative approaches. Extensive experiments have been conducted to compare these approaches under five standard non-IID FL settings, providing much needed insights into which approaches are advantageous under which settings. The proposed framework offers useful guidance on the suitability of various data divergence measures in FL systems. It is beneficial for keeping related research activities on the right track in terms of: (1) designing PFL schemes, (2) selecting appropriate data heterogeneity evaluation approaches for specific FL application scenarios, and (3) addressing fairness issues in collaborative model training. The code is available at https://github.com/Xiaoni-61/DH-Benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。