用方向导数压缩梯度，省带宽还保隐私。
FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivative
- 用多方向导数编码梯度，通信量从O(d)降到O(m)
- 收敛速度达O(1/√K)，与标准联邦学习相当
- 投影数可调，实现隐私与精度的平衡
本文提出FedMPDD（基于多投影方向导数的联邦学习），通过在多个随机向量上计算客户端高维梯度的方向导数，将梯度压缩为小尺寸消息，使上行通信开销从O(d)降至O(m)，其中m远小于d。服务器通过将聚合信息投影回相同随机向量完成解码。关键洞察是：对多个投影取平均，可克服单个投影带来的维数依赖收敛限制。理论分析表明，FedMPDD收敛速率可达O(1/√K)，与FedSGD性能相当。此外，由于低秩投影的几何特性，该方法对梯度反演攻击具有内在隐私保护能力，隐私-效用权衡可通过投影数量调节。在基准数据集上的大量实验验证了理论结果与性能优势。
原文摘要 · Abstract (English)
This paper introduces \texttt{FedMPDD} (\textbf{Fed}erated Learning via \textbf{M}ulti-\textbf{P}rojected \textbf{D}irectional \textbf{D}erivatives), a novel algorithm that simultaneously optimizes bandwidth utilization and enhances privacy in Federated Learning. The core idea of \texttt{FedMPDD} is to encode each client's high-dimensional gradient by computing its directional derivatives along multiple random vectors. This compresses the gradient into a much smaller message, significantly reducing uplink communication costs from $\mathcal{O}(d)$ to $\mathcal{O}(m)$, where $m \ll d$. The server then decodes the aggregated information by projecting it back onto the same random vectors. Our key insight is that averaging multiple projections overcomes the dimension-dependent convergence limitations of a single projection. We provide a rigorous theoretical analysis, establishing that \texttt{FedMPDD} converges at a rate of $\mathcal{O}(1/\sqrt{K})$, matching the performance of FedSGD. Furthermore, we demonstrate that our method provides some inherent privacy against gradient inversion attacks due to the geometric properties of low-rank projections, offering a tunable privacy-utility trade-off controlled by the number of projections. Extensive experiments on benchmark datasets validate our theory and demonstrates our results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。