针对异质协变量下的隐私LASSO，提出非均匀扰动方法提升稳定性与效率。
Stabilizing Private LASSO under Heterogeneous Covariates via Anisotropic Objective Perturbation
- 通过构造基于Gram矩阵的非均匀扰动，抵消协变量尺度差异的影响
- 相比均匀噪声注入，收敛更稳定，统计效率与隐私性能显著提升
- 无需依赖数据预处理，适合高维隐私学习场景
我们研究在差分隐私约束下，针对协变量尺度异质性的高维LASSO问题，采用目标扰动方法。实际中协变量常呈现多样化尺度，但标准预处理在隐私约束下会消耗额外隐私预算，导致问题。这种异质性使目标扰动产生有效各向异性，降低算法稳定性和准确性。为此，我们提出基于Gram矩阵的各向异性目标扰动，即一种“预畸变”策略，以对抗协变量结构带来的失真，恢复估计过程中的各向同性。利用近似消息传递（AMP）框架与状态演化分析，我们证明该扰动能显著稳定收敛，并优于标准均匀噪声注入，在统计效率和隐私性能上均有提升。研究为设计无需依赖数据相关预处理的稳定高效私有估计器提供了理论依据。
原文摘要 · Abstract (English)
We study high-dimensional LASSO under differential privacy via objective perturbation with heterogeneous covariate scales. In practical scenarios, covariates often exhibit diverse scales; however, standard preprocessing is problematic under privacy constraints, as it consumes additional privacy budget. This heterogeneity induces effective anisotropy in the objective perturbation via the inverse Gram matrix of covariates, which can degrade the stability and accuracy of algorithms. To address this, we propose a Gram-based anisotropic objective perturbation, a ``pre-distortion" strategy that counteracts the distortion from the covariate structure to restore isotropy in the estimation process. Using an Approximate Message Passing (AMP) framework and state evolution analysis, we demonstrate that our proposed perturbation significantly stabilizes convergence and improves both statistical efficiency and privacy performance compared to standard uniform noise injection. Our results provide theoretical insights into designing stable and efficient private estimators without relying on data-dependent preprocessing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。