用条件生成器替代特征提取器，实现高隐私保护与高性能的联邦学习
Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generator
- 将客户端模型拆分为生成器和分类器，服务器用生成器聚合
- 通过多层级知识蒸馏保持生成器与分类器的一致性
- 无需额外判别器，对异构数据鲁棒，适合隐私敏感场景
联邦学习（FL）作为一种分布式学习框架，仅共享模型参数或梯度更新，本地保留私有数据，但面临隐私推断攻击风险。现有隐私保护机制常与性能和效率冲突。为此，我们提出FedMD-CG，一种兼具高性能与高水平隐私保护的新型联邦学习方法。该方法将每个客户端的本地模型解耦为特征提取器与分类器，并用条件生成器替代特征提取器进行服务器端模型聚合。为确保本地生成器与分类器的一致性，FedMD-CG在潜在特征层与logit层均采用知识蒸馏训练。此外，构建了附加分类损失并设计新多样性损失以增强客户端训练。实验在多种图像分类任务上验证了其优越性，且对数据异构性具有鲁棒性，无需训练额外判别器（如cGAN）。
原文摘要 · Abstract (English)
Federated Learning (FL) is gaining popularity as a distributed learning framework that only shares model parameters or gradient updates and keeps private data locally. However, FL is at risk of privacy leakage caused by privacy inference attacks. And most existing privacy-preserving mechanisms in FL conflict with achieving high performance and efficiency. Therefore, we propose FedMD-CG, a novel FL method with highly competitive performance and high-level privacy preservation, which decouples each client's local model into a feature extractor and a classifier, and utilizes a conditional generator instead of the feature extractor to perform server-side model aggregation. To ensure the consistency of local generators and classifiers, FedMD-CG leverages knowledge distillation to train local models and generators at both the latent feature level and the logit level. Also, we construct additional classification losses and design new diversity losses to enhance client-side training. FedMD-CG is robust to data heterogeneity and does not require training extra discriminators (like cGAN). We conduct extensive experiments on various image classification tasks to validate the superiority of FedMD-CG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。