高维分布式数据下实现隐私保护的分位数回归与统计推断
High-Dimensional Differentially Private Quantile Regression: Distributed Estimation and Statistical Inference
- 用牛顿变换将分位数损失转为普通最小二乘,解决非光滑难题
- 算法在高维下保持近优统计精度,同时满足差分隐私
- 支持置信区间构建与多假设检验,适合小样本或大数据场景
随着大数据和机器学习的发展,处理包含敏感个人信息的异构数据集时,隐私保护日益重要。差分隐私为在保障个体隐私的同时进行有意义的统计分析提供了严格框架。本文提出一种适用于高维分布式数据的差分隐私分位数回归方法。分位数回归是建模协变量与响应变量关系的强大且稳健工具,尤其在存在异常值或重尾分布时表现优异。为应对分位数损失函数非光滑带来的计算挑战,我们引入一种牛顿型变换,将分位数回归任务重构为普通最小二乘问题。基于此,我们设计了一种具有迭代更新的差分隐私估计算法,实现了近优统计精度与形式化隐私保障。针对推断需求,进一步提出差分隐私去偏估计器,支持有效置信区间构造与假设检验。此外,我们还提出一种通信高效且差分隐私的自助法,适用于高维分位数回归中的联合假设检验,特别适配本地数据量小或丰富的分布式场景。大量模拟实验验证了所提方法在实际场景中的鲁棒性与有效性。
原文摘要 · Abstract (English)
With the development of big data and machine learning, privacy concerns have become increasingly critical, especially when handling heterogeneous datasets containing sensitive personal information. Differential privacy provides a rigorous framework for safeguarding individual privacy while enabling meaningful statistical analysis. In this paper, we propose a differentially private quantile regression method for high-dimensional data in a distributed setting. Quantile regression is a powerful and robust tool for modeling the relationships between the covariates and responses in the presence of outliers or heavy-tailed distributions. To address the computational challenges due to the non-smoothness of the quantile loss function, we introduce a Newton-type transformation that reformulates the quantile regression task into an ordinary least squares problem. Building on this, we develop a differentially private estimation algorithm with iterative updates, ensuring both near-optimal statistical accuracy and formal privacy guarantees. For inference, we further propose a differentially private debiased estimator, which enables valid confidence interval construction and hypothesis testing. Additionally, we propose a communication-efficient and differentially private bootstrap for simultaneous hypothesis testing in high-dimensional quantile regression, suitable for distributed settings with both small and abundant local data. Extensive simulations demonstrate the robustness and effectiveness of our methods in practical scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。