用PAC隐私保护模型输出，实现高精度低泄漏的私有预测。
Private Prediction via PAC Privacy
- 基于实例稳定性的噪声自适应校准，提升隐私效率。
- 百万级查询下成员推断成功率仅51.08%，媲美差分隐私。
- 可生成可公开查询的轻量模型，适合实际服务部署。
机器学习模型越来越多通过API提供服务，私有预测（即对模型输出而非参数进行隐私保护）成为自然目标：输出维度更低且对训练数据变化更稳定。尽管差分隐私（DP）难以利用这一优势，因其噪声需按最坏情况敏感度校准，而该值对非凸模型难以界定，我们提出采用PAC隐私更合适。它基于实例，将噪声校准至黑箱函数的经验稳定性，以控制互信息（MI）泄露。现有组合机制在自适应查询下失效或噪声增长过快，或退化为与输入无关的DP式噪声。本文提出新型对抗性组合结果，通过自适应噪声校准，证明在自适应和对抗查询下MI仅线性累积。多模态实验表明，预测稳定性使极小每查询隐私预算下仍保持高精度：在CIFAR-10上达87.79%准确率，每查询MI预算为2⁻³²；支持一百万次查询，成员推断成功率被严格限制在51.08%——等效于(0.04, 10⁻⁵)-DP。此外，在存在辅助公开数据时，大量PAC私有预测可用于蒸馏出可无限查询的发布模型：21万条ImageNet子集标签可蒸馏出学生模型在CIFAR-10上达91.86%准确率，成员推断成功率控制在50.49%，接近(0.02, 10⁻⁵)-DP性能。
原文摘要 · Abstract (English)
Machine learning models are increasingly served behind APIs. This renders private prediction, i.e., privatizing a model's outputs rather than its parameters, a natural privacy target: model outputs are lower-dimensional and far more stable to training-data changes than weights. While differential privacy (DP) cannot effectively exploit this as it calibrates noise to worst-case sensitivity that is intractable to bound for non-convex models, we argue that PAC privacy is a natural fit for private prediction. It is instance-based, and calibrates noise to a black-box function's empirical stability to control mutual-information (MI) leakage. The missing ingredient is efficient, adaptive composition. Serving predictions means answering a long stream of adaptively chosen queries from untrusted users; existing composition either fails under adaptivity, grows quadratically, or reverts to input-independent, DP-like noise. We close this gap with a new adversarial composition result via adaptive noise calibration and prove that MI accumulates only linearly under adaptive and adversarial querying. Experiments across modalities show that prediction stability enables high utility even at a tiny per-query budget: on CIFAR-10, we achieve 87.79% accuracy with a per-query MI budget of $2^{-32}$. This enables serving one million queries while provably bounding membership-inference success to 51.08% -- the same guarantee as $(0.04, 10^{-5})$-DP. Further, in the presence of auxiliary public data, the large volume of PAC-private predictions enables us to distill a publishable model that can be queried without limit. Concretely, 210,000 private labels on an ImageNet subset distill into a student reaching 91.86% accuracy on CIFAR-10 with membership inference success bounded by 50.49%, comparable to $(0.02, 10^{-5})$-DP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。