提出无需梯度的联邦学习框架,实现低通信、高隐私与可加密预测。
Operator-Theoretic Framework for Gradient-Free Federated Learning
- 用算子理论将最优解映射到核空间,通过数据近似实现无梯度更新。
- 在4个基准上性能媲美甚至超越梯度方法,最高提升23.7分。
- 支持差分隐私与全同态加密,适合资源受限且注重隐私的场景。
联邦学习需应对异构性、严苛的通信与计算限制及隐私保护,同时保证性能。本文提出一种算子理论框架:通过前向算子将$L^2$最优解映射至再生核希尔伯特空间(RKHS),利用可用数据近似后,再经逆算子映射回原空间,得到无梯度方案。基于算子范数上的集中不等式,推导出有限样本界,识别出具有风险、误差、鲁棒性与逼近保证的数据相关假设空间。在此空间中设计高效核机器,利用核仿射壳机的空间折叠特性。客户端通过标量空间折叠度量传递知识,降低通信开销,并实现简单差分隐私协议:仅需一步对加噪数据矩阵计算摘要,避免逐轮裁剪与隐私会计。全局规则每测试点仅需整数最小值和等值比较操作,兼容全同态加密(FHE)。在四个基准上,使用固定编码嵌入的无梯度联邦学习方法表现匹配或优于强梯度微调,最高提升23.7个百分点。在差分隐私实验中,核平滑有效缓解高隐私设置下的精度损失。全局规则可实现为$Q \times C$个加密最小值与$C$个等值比较操作,操作级基准显示实际延迟可行。整体框架提供可证明保证,低通信,支持隐私知识迁移,并生成适用于异构环境的数学严谨型联邦学习替代方案。
原文摘要 · Abstract (English)
Federated learning must address heterogeneity, strict communication and computation limits, and privacy while ensuring performance. We propose an operator-theoretic framework that maps the $L^2$-optimal solution into a reproducing kernel Hilbert space (RKHS) via a forward operator, approximates it using available data, and maps back with the inverse operator, yielding a gradient-free scheme. Finite-sample bounds are derived using concentration inequalities over operator norms, and the framework identifies a data-dependent hypothesis space with guarantees on risk, error, robustness, and approximation. Within this space we design efficient kernel machines leveraging the space folding property of Kernel Affine Hull Machines. Clients transfer knowledge via a scalar space folding measure, reducing communication and enabling a simple differentially private protocol: summaries are computed from noise-perturbed data matrices in one step, avoiding per-round clipping and privacy accounting. The induced global rule requires only integer minimum and equality-comparison operations per test point, making it compatible with fully homomorphic encryption (FHE). Across four benchmarks, the gradient-free FL method with fixed encoder embeddings matches or outperforms strong gradient-based fine-tuning, with gains up to 23.7 points. In differentially private experiments, kernel smoothing mitigates accuracy loss in high-privacy regimes. The global rule admits an FHE realization using $Q \times C$ encrypted minimum and $C$ equality-comparison operations per test point, with operation-level benchmarks showing practical latencies. Overall, the framework provides provable guarantees with low communication, supports private knowledge transfer via scalar summaries, and yields an FHE-compatible prediction rule offering a mathematically grounded alternative to gradient-based federated learning under heterogeneity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。