arXiv:2502.20565cs.LGcs.AI2025-02

用零阶优化实现低通信量、高隐私的垂直联邦学习。

Communication-Efficient and Differentially Private Vertical Federated Learning with Zeroth-Order Optimization

  • 用零阶优化在下行通信中注入微调的差分隐私噪声,降低梯度质量损失。
  • 在严格隐私约束(ε≤10)下,通信轮次更少,隐私-效用权衡更优。
  • 适合对通信成本和数据隐私要求高的跨机构协作场景。

垂直联邦学习(VFL)使特征分区设备间可协同训练模型,但依赖设备-服务器信息交换带来显著通信开销与隐私风险。现有方案在下行通信中注入差分隐私(DP)噪声,导致梯度质量下降、收敛变慢、通信轮次过多。本文提出DPZV框架,基于零阶(ZO)优化,在下行通信中注入校准的标量级DP噪声,显著减少方差放大,同时提供对抗目标推理攻击的有效保护。理论分析表明,尽管仅使用零阶估计器,其收敛性仍可媲美一阶DP-SGD,并满足(ε, δ)-DP。大量实验显示,当ε≤10时,DPZV在隐私-效用权衡上优于现有基线,且通信轮次更少。

原文摘要 · Abstract (English)

Vertical Federated Learning (VFL) enables collaborative model training across feature-partitioned devices, yet its reliance on device-server information exchange introduces significant communication overhead and privacy risks. Downlink communication from the server to devices in VFL exposes gradient-related signals of the global loss that can be leveraged in inference attacks. Existing privacy-preserving VFL approaches that inject differential privacy (DP) noise on the downlink have the natural repercussion of degraded gradient quality, slowed convergence, and excessive communication rounds. In this work, we propose DPZV, a communication-efficient and differentially private ZO-VFL framework with tunable privacy guarantees. Based on zeroth-order (ZO) optimization, DPZV injects calibrated scalar-valued DP noise on the downlink, significantly reducing variance amplification while providing equivalent protection against targeted inference attacks. Through rigorous theoretical analysis, we establish convergence guarantees comparable to first-order DP-SGD, despite relying solely on ZO estimators, and prove that DPZV satisfies $(ε, δ)$-DP. Extensive experiments demonstrate that DPZV consistently achieves a superior privacy-utility tradeoff and requires fewer communication rounds than existing DP-VFL baselines under strict privacy constraints ($ε\leq 10$).

联邦学习差分隐私通信效率零阶优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。