在密集数据分布下,实现高效且最优的自适应数据分析。
Efficient Adaptive Data Analysis over Dense Distributions
- 针对密集分布设计高效算法,突破计算与样本复杂度的权衡
- 达到最优 $O(\log T)$ 样本复杂度,优于传统 $O(\sqrt{T})$
- 适用于特定分布学习,兼具统计查询效率与松散隐私保障
现代数据工作流具有内在的自适应性,会反复查询同一数据集以优化和验证决策,但这种自适应性可能导致过拟合和无效的统计推断。自适应数据分析(ADA)机制旨在应对这一挑战,然而计算效率与样本复杂度之间存在根本矛盾:对于 $T$ 轮自适应分析,计算高效的算法通常仅达到次优的 $O(\sqrt{T})$ 样本复杂度,而统计最优的 $O(\log T)$ 算法在标准密码学假设下是计算不可行的。本文揭示了这一权衡的内在机制,识别出一类自然的数据分布,在该分布下可同时实现计算效率与最优样本复杂度。我们提出一种计算高效的 ADA 机制,当数据分布相对于已知先验为密集时,可实现最优的 $O(\log T)$ 样本复杂度。该设定包括特定分布学习中的特征-标签分布。因此,我们的机制也给出了分布特定设置下的样本高效($O(\log T)$)统计查询预言机。此外,尽管算法不基于差分隐私,但满足一种宽松的隐私概念——谓词单点攻击(PSO)安全性(Cohen and Nissim, 2020)。结果表明,自适应数据分析与隐私之间存在超越差分隐私的深层联系。
原文摘要 · Abstract (English)
Modern data workflows are inherently adaptive, repeatedly querying the same dataset to refine and validate sequential decisions, but such adaptivity can lead to overfitting and invalid statistical inference. Adaptive Data Analysis (ADA) mechanisms address this challenge; however, there is a fundamental tension between computational efficiency and sample complexity. For $T$ rounds of adaptive analysis, computationally efficient algorithms typically incur suboptimal $O(\sqrt{T})$ sample complexity, whereas statistically optimal $O(\log T)$ algorithms are computationally intractable under standard cryptographic assumptions. In this work, we shed light on this trade-off by identifying a natural class of data distributions under which both computational efficiency and optimal sample complexity are achievable. We propose a computationally efficient ADA mechanism that attains optimal $O(\log T)$ sample complexity when the data distribution is dense with respect to a known prior. This setting includes, in particular, feature--label data distributions arising in distribution-specific learning. As a consequence, our mechanism also yields a sample-efficient (i.e., $O(\log T)$ samples) statistical query oracle in the distribution-specific setting. Moreover, although our algorithm is not based on differential privacy, it satisfies a relaxed privacy notion known as Predicate Singling Out (PSO) security (Cohen and Nissim, 2020). Our results thus reveal an inherent connection between adaptive data analysis and privacy beyond differential privacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。