arXiv:2503.11897cs.CRcs.DS2025-03NeurIPS被引 1

用块稀疏向量实现高效私有聚合,通信成本大幅降低。

PREAMBLE: Private and Efficient Aggregation via Block Sparse Vectors

  • 基于块稀疏分布式点函数,实现低通信开销的向量聚合。
  • 在保持隐私保护的前提下,噪声方差接近高斯机制最优水平。
  • 适合大规模私有联邦学习中的梯度聚合场景。

我们重新审视双服务器系统(如 Prio)中高维向量的隐私聚合问题。这类系统常用于私有联邦学习中的梯度聚合,通过添加噪声保证差分隐私。现有方法通信开销随维度线性增长,限制了可处理向量的维度。本文提出 PREAMBLE:一种基于块稀疏欧几里得向量的私有高效聚合机制。该方法扩展了分布式点函数,支持块稀疏向量的通信与计算高效聚合——即非零元素集中在少数连续坐标块中。结合随机采样与采样隐私放大技术,实现了渐近最优的隐私-效用权衡,且通信成本仅为原有方案的一小部分。结合近期数值隐私会计进展,其噪声方差相比 Prio 中使用的高斯机制仅增加可忽略的开销。

原文摘要 · Abstract (English)

We revisit the problem of secure aggregation of high-dimensional vectors in a two-server system such as Prio. These systems are typically used to aggregate vectors such as gradients in private federated learning, where the aggregate itself is protected via noise addition to ensure differential privacy. Existing approaches require communication scaling with the dimensionality, and thus limit the dimensionality of vectors one can efficiently process in this setup. We propose PREAMBLE: {\bf Pr}ivate {\bf E}fficient {\bf A}ggregation {\bf M}echanism via {\bf BL}ock-sparse {\bf E}uclidean Vectors. PREAMBLE builds on an extension of distributed point functions that enables communication- and computation-efficient aggregation of {\em block-sparse vectors}, which are sparse vectors where the non-zero entries occur in a small number of clusters of consecutive coordinates. We show that these block-sparse DPFs can be combined with random sampling and privacy amplification by sampling results, to allow asymptotically optimal privacy-utility trade-offs for vector aggregation, at a fraction of the communication cost. When coupled with recent advances in numerical privacy accounting, our approach incurs a negligible overhead in noise variance, compared to the Gaussian mechanism used with Prio.

隐私聚合联邦学习块稀疏差分隐私

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。