提出隐私保护的两阶段梯度下降法,解决工具变量回归的隐私泄露问题。
Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression
- 通过在梯度更新中注入精确校准的噪声,实现ρ-零集中差分隐私
- 理论证明算法在有限样本下收敛,且保持一致性
- 首次为线性模型下的工具变量回归提供隐私保障与收敛率保证
我们在差分隐私约束下研究工具变量回归(IVaR)。传统IVaR方法(如两阶段最小二乘)依赖于直接使用敏感协变量和工具变量的矩方程,存在显著的隐私泄露风险,且难以设计兼具统计效率与隐私保护的算法。本文提出一种带噪声的两阶段梯度下降算法,通过在梯度更新中注入精心校准的噪声,实现ρ-零集中差分隐私。理论分析给出了该方法的有限样本收敛速率,表明算法在保证隐私的同时具有相合性。特别地,我们推导了优化、隐私与抽样误差之间权衡的精确边界。据我们所知,这是首个在线性模型中为工具变量回归同时提供隐私保障与可证明收敛率的工作。我们在合成数据和真实数据集上进行了实验验证,结果表明该方法能实现有效的精度-隐私权衡。
原文摘要 · Abstract (English)
We study instrumental variable regression (IVaR) under differential privacy constraints. Classical IVaR methods (like two-stage least squares regression) rely on solving moment equations that directly use sensitive covariates and instruments, creating significant risks of privacy leakage and posing challenges in designing algorithms that are both statistically efficient and differentially private. We propose a noisy two-stage gradient descent algorithm that ensures $ρ$-zero-concentrated differential privacy by injecting carefully calibrated noise into the gradient updates. Our analysis establishes finite-sample convergence rates for the proposed method, showing that the algorithm achieves consistency while preserving privacy. In particular, we derive precise bounds quantifying the trade-off among optimization, privacy, and sampling error. To the best of our knowledge, this is the first work to provide both privacy guarantees and provable convergence rates for instrumental variable regression in linear models. We further validate our theoretical findings with experiments on both synthetic and real datasets, demonstrating that our method offers practical accuracy-privacy trade-offs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。