arXiv:2412.07093cs.LGcs.CR2024-12中稿 · SaTML 2025被引 9

用分箱法在低内存下实现差分隐私的持续计数,精度接近最优。

Streaming Private Continual Counting via Binning

  • 通过分箱将相似矩阵元素合并,实现子线性空间的矩阵向量积维护
  • 在极低内存下性能接近甚至超过现有最优因子分解机制
  • 适用于需要持续隐私保护的在线学习场景,如差分隐私梯度下降

在差分隐私中,持续观测指需随数据逐条流入持续发布函数值的问题。其挑战在于保持良好近似的同时,保证所有时间步输出的联合隐私性。在持续计数这一特例中,目标是近似二元输入元素之和。因子分解机制是当前主流方法,但其空间复杂度与输入规模成正比,不适用于流式场景。本文提出一种基于分箱的简化方法,在低空间内逼近因子分解机制:将相邻且值相近的矩阵元素设为相同,从而以子线性空间维护矩阵-向量乘积。该方法对一类沿对角线单调递减的下三角矩阵具有可证明的子线性空间保证。实验表明,即使使用极低空间,也能接近甚至超越渐进最优因子分解机制的性能。近期独立工作(Dvijotham等)也提出了流式实现方案,但仅针对托普利茨矩阵、仅关注最大误差,且采用基于有理函数逼近的非通用技术,相较之下本方法更具灵活性。

原文摘要 · Abstract (English)

In differential privacy, $\textit{continual observation}$ refers to problems in which we wish to continuously release a function of a dataset that is revealed one element at a time. The challenge is to maintain a good approximation while keeping the combined output over all time steps differentially private. In the special case of $\textit{continual counting}$ we seek to approximate a sum of binary input elements. This problem has received considerable attention lately, in part due to its relevance in implementations of differentially private stochastic gradient descent. $\textit{Factorization mechanisms}$ are the leading approach to continual counting, but the best such mechanisms do not work well in $\textit{streaming}$ settings since they require space proportional to the size of the input. In this paper, we present a simple approach to approximating factorization mechanisms in low space via $\textit{binning}$, where adjacent matrix entries with similar values are changed to be identical in such a way that a matrix-vector product can be maintained in sublinear space. Our approach has provable sublinear space guarantees for a class of lower triangular matrices whose entries are monotonically decreasing away from the diagonal. We show empirically that even with very low space usage we are able to closely match, and sometimes surpass, the performance of asymptotically optimal factorization mechanisms. Recently, and independently of our work, Dvijotham et al. have also suggested an approach to implementing factorization mechanisms in a streaming setting. Their work differs from ours in several respects: It only addresses factorization into $\textit{Toeplitz}$ matrices, only considers $\textit{maximum}$ error, and uses a different technique based on rational function approximation that seems less versatile than our binning approach.

差分隐私持续计数流式计算分箱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。