打造可复现的联邦学习评测平台,支持模拟与真实部署统一测试
Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment

- 将联邦学习基准封装为带版本和依赖的可执行应用
- 覆盖医疗影像等5大领域,支持跨场景统一评估
- 同一代码可在模拟与真实环境运行,提升评测实用性
联邦学习(FL)已成为在分散数据上训练模型的关键方法,但现有评估难以复现、比较和扩展。多数工作依赖定制基础设施,代码不完整,且仅在模拟环境中进行,限制了可移植性和实际意义。我们提出 Flower Hub,一个用于发布、发现和执行去中心化及联邦应用的平台。通过将基准封装为带标准化元数据、固定依赖和明确评估流程的可执行应用,实现了可复现的基准测试。我们构建了一个涵盖跨孤岛与跨设备场景的多领域基准套件,包含医学影像、金融表格学习、法律指令微调、钓鱼网址检测和音频标记等任务。进一步证明,相同基准应用无需修改代码即可在模拟与部署环境中运行,实现跨学习环境的统一评估。除模型性能外,该设计还支持系统级指标报告,如运行时和通信开销。本工作推动联邦学习基准从临时代码向可移植、可执行、可重用的应用转变。
原文摘要 · Abstract (English)
Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reproduce, compare, and extend. Existing evaluations are often tied to custom infrastructure, released as incomplete research code, and conducted primarily in simulation, which limits portability and practical relevance. We present Flower Hub, a platform for publishing, discovering, and executing decentralized and federated applications. We show how it enables reproducible benchmarking by packaging benchmarks as executable, versioned applications with standardized metadata, pinned dependencies, and explicit evaluation workflows. We instantiate this approach with a multi-domain benchmark suite spanning cross-silo and cross-device settings, and including tasks in medical imaging, financial tabular learning, legal instruction tuning, phishing URL detection, and audio tagging. We further demonstrate that the same benchmarking application can run across both simulation and deployment runtimes without changing the application code, enabling unified evaluation across varying learning environments. Beyond model quality, our benchmark design supports system-aware reporting, including runtime and communication metrics. This work advances benchmarking in FL settings from ad hoc code artifacts towards portable, executable, and reusable benchmark applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。