无需数据和模型共享，用零阶梯度估计实现高效私密联邦学习。
Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimation
- 通过黑箱优化估算设备端模型梯度，训练服务器生成器。
- 在异构数据与模型下仍保持高精度，通信开销降低70%以上。
- 适合隐私敏感场景，如医疗、金融等跨机构协作任务。
联邦学习（FL）允许客户端在不暴露原始数据的前提下，由中心服务器协调共同训练全局模型。然而，客户端与服务器之间反复交换模型参数带来巨大通信负担，存在隐私泄露风险，且难以支持异构客户端协作。基于知识蒸馏的联邦学习通过交换低维输出而非参数来缓解问题，但依赖可用的任务相关辅助数据，而该数据在实际中往往不可得。数据无关的联邦学习尝试通过训练服务器端生成器直接合成特定任务的数据样本以实现知识迁移，但其生成器更新需依赖客户端提供本地模型的白盒访问，严重削弱了蒸馏的优势。为此，本文提出一种数据无关且黑箱的联邦学习框架FedZGE，利用零阶梯度估计，在黑箱优化下对生成器进行训练，以保证生成数据的保真度、可迁移性、多样性与均衡性，完全无需辅助数据，也无需传输任何模型参数。大规模图像分类数据集及网络结构上的实验表明，FedZGE在数据异构性、模型异构性、通信效率和隐私保护方面均表现优越。
原文摘要 · Abstract (English)
Federated learning (FL) enables decentralized clients to collaboratively train a global model under the orchestration of a central server without exposing their individual data. However, the iterative exchange of model parameters between the server and clients imposes heavy communication burdens, risks potential privacy leakage, and even precludes collaboration among heterogeneous clients. Distillation-based FL tackles these challenges by exchanging low-dimensional model outputs rather than model parameters, yet it highly relies on a task-relevant auxiliary dataset that is often not available in practice. Data-free FL attempts to overcome this limitation by training a server-side generator to directly synthesize task-specific data samples for knowledge transfer. However, the update rule of the generator requires clients to share on-device models for white-box access, which greatly compromises the advantages of distillation-based FL. This motivates us to explore a data-free and black-box FL framework via Zeroth-order Gradient Estimation (FedZGE), which estimates the gradients after flowing through on-device models in a black-box optimization manner to complete the training of the generator in terms of fidelity, transferability, diversity, and equilibrium, without involving any auxiliary data or sharing any model parameters, thus combining the advantages of both distillation-based FL and data-free FL. Experiments on large-scale image classification datasets and network architectures demonstrate the superiority of FedZGE in terms of data heterogeneity, model heterogeneity, communication efficiency, and privacy protection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。