解决多个可复现算法联合运行的样本复杂度问题,实现最优性能。
Replicable Composition
- 将可复现算法转化为完美泛化形式,通过隐私分析组合并回溯。
- 实现仅需总样本量的对数级开销,达到理论最优的O~(∑ni)。
- 适用于多类问题,尤其适合对成功率有严格要求的研究者。
可复现性要求算法在独立采样的数据上重复运行时结论一致。核心问题是组合:给定k个问题,每个问题存在ρ-可复现算法且样本复杂度为n,联合求解所有问题并保持可复现性需要多少样本?朴素分析得 ilde{O}(nk^2),Bun等(STOC'23)通过差分隐私归约得到 ilde{O}(n^2k),但最优 ilde{O}(nk)是否可达仍悬而未决。本文解决该问题,更一般地证明:样本复杂度为n₁,…,nₖ的问题可联合求解,仅需 ilde{O}(∑ᵢ nᵢ)样本且保持常数可复现性。方法是将每个可复现算法转为完美泛化形式,经隐私风格分析组合,再用相关采样映射回原问题。首次建立可复现性的高级组合定理。过程中还获得针对参数异构的完美泛化算法组合新界。此外,提出可复现算法成功率的提升定理,对一大类问题,失败概率以独立于ρ的加性项出现,直接改进多个问题的样本复杂度。最后证明自适应组合下Ω(nk²)下界,与非自适应情形形成二次分离。关键技巧‘虚拟运行’带来具有独立意义的结构性结果。
原文摘要 · Abstract (English)
Replicability requires that algorithmic conclusions remain consistent when rerun on independently drawn data. A central structural question is composition: given $k$ problems each admitting a $ρ$-replicable algorithm with sample complexity $n$, how many samples are needed to solve all jointly while preserving replicability? The naive analysis yields $\widetilde{O}(nk^2)$ samples, and Bun et al. (STOC'23) observed that reductions through differential privacy give an alternative $\widetilde{O}(n^2k)$ bound, leaving open whether the optimal $\widetilde{O}(nk)$ scaling is achievable. We resolve this open problem and, more generally, show that problems with sample complexities $n_1,\ldots,n_k$ can be jointly solved with $\widetilde{O}(\sum_i n_i)$ samples while preserving constant replicability. Our approach converts each replicable algorithm into a perfectly generalizing one, composes them via a privacy-style analysis, and maps back via correlated sampling. This yields the first advanced composition theorem for replicability. En route, we obtain new bounds for the composition of perfectly generalizing algorithms with heterogeneous parameters. As part of our results, we provide a boosting theorem for the success probability of replicable algorithms. For a broad class of problems, the failure probability appears as a separate additive term independent of $ρ$, immediately yielding improved sample complexity bounds for several problems. Finally, we prove an $Ω(nk^2)$ lower bound for adaptive composition, establishing a quadratic separation from the non-adaptive setting. The key technique, which we call the phantom run, yields structural results of independent interest.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。