SSTQ用低通信量实现隐私保护,比现有方法更高效
SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant

- 用随机采样与压缩框架降低量化误差
- 每客户端仅需⌈log₂N⌉+b比特,误差随维度最优增长
- 适合联邦学习中对隐私和带宽敏感的场景
在分布式优化中实现本地差分隐私并保持低通信开销仍具挑战。现有向量量化方法如vqSGD依赖高维几何结构,但导致不利的维度相关方差。本文提出子采样随机涡轮量化(SSTQ),结合过完备等范紧框架、坐标子采样与隐私感知一维量化。SSTQ包含两种变体:扁平随机响应版与度量感知拉普拉斯版,后者更适合更高码本位宽情形。我们证明SSTQ实现最优均方误差缩放,每客户端仅使用⌈log₂N⌉+b比特,其中N=Θ(d)为框架大小。我们还推导出一种代理隐私感知码本目标函数,将码本相关均方误差缩放从O(4^b)降至O(2^b)。我们在CIFAR-10和Fashion-MNIST上对联邦学习任务进行实验评估,结果表明SSTQ在实用性与通信效率上优于基准方法。部分分析推导由谷歌内部开发的全自动化Gemini智能体系统首次获得,作者已验证并润色以提升表述清晰度。
原文摘要 · Abstract (English)
Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging. Existing vector quantization methods, such as vqSGD, use high-dimensional geometric constructions but incur unfavorable dimension-dependent variance. In this work, we propose Subsampled Stochastic TurboQuant (SSTQ), a framework that combines overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization. SSTQ includes two variants: a Flat Randomized Response version and a Metric-Aware Laplace version, the latter being better suited to higher codebook bit-width regimes. We show that SSTQ achieves optimal mean squared error scaling while using only $\lceil \log_2 N \rceil + b$ bits per client, where $N = Θ(d)$ is the frame size. We also derive a surrogate privacy-aware codebook objective that reduces the codebook-dependent MSE scaling from $O(4^b)$ to $O(2^b)$. Finally, we empirically evaluate SSTQ against established baselines on federated learning tasks using CIFAR-10 and Fashion-MNIST, demonstrating favorable utility and communication efficiency. Some of the analytical derivations were first obtained using a fully automated Gemini-based agentic system developed internally at Google. The authors have verified those derivations and edited them for clarity of presentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。