用傅里叶变换分离隐私噪声，提升差分隐私训练的准确率。
Fast Fourier Transform-Based Spectral and Temporal Gradient Filtering for Differential Privacy
- 在频域过滤梯度，将噪声集中到低信息量高频部分，保留重要低频信号。
- 结合卡尔曼滤波与有限差分海森矩阵，进一步降低噪声并提升梯度质量。
- 适用于图像分类任务，对CNN、ViT等模型均有效，显著优于传统DP方法。
差分隐私（DP）已成为机器学习中保护敏感数据的关键框架，但标准的DP-SGD常因注入噪声导致显著精度损失。为解决此问题，我们提出基于快速傅里叶变换的增强型卡尔曼滤波器（FFTKF），一种保持$(\varepsilon, δ)$-DP保证的同时提升梯度质量的差分隐私优化方法。FFTKF在频域对梯度进行过滤，将隐私噪声转移至信息量较低的高频成分，从而保留携带主要学习信息的低频梯度信号。通过采用带有限差分海森矩阵近似的标量增益卡尔曼滤波器，进一步精炼去噪后的梯度。该方法每轮迭代复杂度为$\mathcal{O}(d \log d)$，在MNIST、CIFAR-10、CIFAR-100和Tiny-ImageNet上使用CNN、Wide ResNets和Vision Transformers均实现了比DP-SGD和DiSK更高的测试准确率。理论分析表明，FFTKF在保持同等隐私水平的同时，通过降低方差和可控偏差，实现了更强的隐私-效用权衡。
原文摘要 · Abstract (English)
Differential Privacy (DP) has emerged as a key framework for protecting sensitive data in machine learning, but standard DP-SGD often suffers from significant accuracy loss due to injected noise. To address this limitation, we introduce the FFT-Enhanced Kalman Filter (FFTKF), a differentially private optimization method that improves gradient quality while preserving $(\varepsilon, δ)$-DP guarantees. FFTKF applies frequency-domain filtering to shift privacy noise into less informative high-frequency components, preserving the low-frequency gradient signals that carry most learning information. A scalar-gain Kalman filter with a finite-difference Hessian approximation further refines the denoised gradients. The method has per-iteration complexity $\mathcal{O}(d \log d)$ and achieves higher test accuracy than DP-SGD and DiSK on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet with CNNs, Wide ResNets, and Vision Transformers. Theoretical analysis shows that FFTKF ensures equivalent privacy while delivering a stronger privacy--utility trade-off through reduced variance and controlled bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。