提出隐私保护的分布式数据蒸馏框架,防止数据泄露和攻击。
Secure Federated Data Distillation
- 在不共享原始数据前提下,各客户端协作生成小型合成数据集。
- 通过梯度匹配与差分隐私防御,确保数据安全且性能损失小。
- 适合医疗、金融等敏感数据场景下的安全模型训练。
数据蒸馏(DD)是一种将大规模数据集压缩为紧凑、代表性合成数据集的强大技术,可加速机器学习训练。然而,传统方法为集中式处理,存在严重隐私风险,限制了其应用。为此,我们提出安全联邦数据蒸馏(SFDD)框架,实现去中心化蒸馏过程并保障隐私。不同于现有联邦蒸馏仅聚焦于用蒸馏知识训练全局模型,我们的方法旨在生成不暴露本地贡献的蒸馏数据集。我们采用基于梯度匹配的数据蒸馏方法,适配分布式环境,使客户端在不共享原始数据的情况下参与蒸馏。中央聚合器通过整合客户端更新迭代优化合成数据集,同时保证数据机密性。为防范服务器利用梯度更新进行推断攻击,我们设计了一种优化的局部差分隐私方法——LDPO-RLD。此外,我们在假设有足够多参与客户端的前提下,评估了框架对恶意客户端发起后门攻击(如Doorping)的鲁棒性。实验结果表明,SFDD有效且所提防御显著缓解了已识别漏洞,对蒸馏数据集性能影响极小。本工作通过解决数据蒸馏中隐私与联邦的协同问题,推进了隐私保护机器学习的发展,使SFDD成为敏感数据共享应用的可行方案。
原文摘要 · Abstract (English)
Dataset Distillation (DD) is a powerful technique for reducing large datasets into compact, representative synthetic datasets, accelerating Machine Learning training. However, traditional DD methods operate in a centralized manner, which poses significant privacy threats and reduces its applicability. To mitigate these risks, we propose a Secure Federated Data Distillation (SFDD) framework to decentralize the distillation process while preserving privacy. Unlike existing Federated Distillation techniques that focus on training global models with distilled knowledge, our approach aims to produce a distilled dataset without exposing local contributions. We leverage the gradient-matching-based distillation method, adapting it for a distributed setting where clients contribute to the distillation process without sharing raw data. The central aggregator iteratively refines a synthetic dataset by integrating client-side updates while ensuring data confidentiality. To make our approach resilient to inference attacks perpetrated by the server that could exploit gradient updates to reconstruct private data, we create an optimized Local Differential Privacy approach, called LDPO-RLD. Furthermore, we assess the framework's resilience against malicious clients executing backdoor attacks (such as Doorping) and demonstrate robustness under the assumption of a sufficient number of participating clients. Our experimental results demonstrate the effectiveness of SFDD and that the proposed defense concretely mitigates the identified vulnerabilities, with minimal impact on the performance of the distilled dataset. By addressing the interplay between privacy and federation in dataset distillation, this work advances the field of privacy-preserving Machine Learning making our SFDD framework a viable solution for sensitive data-sharing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。