用控制理论方法保护联邦学习隐私,不降性能也不减效率。
Immersion and Invariance-based Coding for Privacy-Preserving Federated Learning
- 将优化算法视为动态系统,通过浸入设计实现隐私编码。
- 可任意调节差分隐私级别,仍保持原模型精度与收敛速度。
- 适合对隐私和性能都有高要求的联邦学习应用场景。
联邦学习(FL)是一种在分布式协作学习中保护隐私的方法,客户端在本地设备上训练模型而不共享数据,但仍可能通过共享的模型更新泄露本地数据信息。尽管已有多种隐私保护方法,但常以牺牲模型性能或系统效率为代价。本文提出一种结合差分隐私与控制理论中浸入不变性(Immersion and Invariance)的隐私保护框架。核心思想是将标准FL中的优化算法(如梯度法)视为动态系统,并将其浸入到一个高维目标优化系统中。该目标系统的动态设计确保:原始算法参数被浸入其参数中;操作于扭曲后的参数;最终收敛至原始参数的编码版本。服务器端可解码得到原始模型参数。实验表明,该方案可灵活设定任意级别的差分隐私(针对本地与全局参数),同时维持与标准FL相同的准确率和收敛速率。
原文摘要 · Abstract (English)
Federated learning (FL) has emerged as a method to preserve privacy in collaborative distributed learning. In FL, clients train AI models directly on their devices rather than sharing data with a centralized server, which can pose privacy risks. However, it has been shown that despite FL's partial protection of local data privacy, information about clients' data can still be inferred from shared model updates during training. In recent years, several privacy-preserving approaches have been developed to mitigate this privacy leakage in FL, though they often provide privacy at the cost of model performance or system efficiency. Balancing these trade-offs presents a significant challenge in implementing FL schemes. In this manuscript, we introduce a privacy-preserving FL framework that combines differential privacy and system immersion tools from control theory. The core idea is to treat the optimization algorithms used in standard FL schemes (e.g., gradient-based algorithms) as a dynamical system that we seek to immerse into a higher-dimensional system (referred to as the target optimization algorithm). The target algorithm's dynamics are designed such that, first, the model parameters of the original algorithm are immersed in its parameters; second, it operates on distorted parameters; and third, it converges to an encoded version of the true model parameters from the original algorithm. These encoded parameters can then be decoded at the server to retrieve the original model parameters. We demonstrate that the proposed privacy-preserving scheme can be tailored to offer any desired level of differential privacy for both local and global model parameters, while maintaining the same accuracy and convergence rate as standard FL algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。