提出自适应大间隔的私有分类算法，提升小异常数据下的模型精度。
Adapting to Linear Separable Subsets with Large-Margin in Differentially Private Learning
- 基于大间隔线性可分性设计私有学习算法，自动适应未知边缘参数和异常点集。
- 在小异常点情况下，误差界比现有方法更优，达到 $\tilde{O}(\frac{1}{γ^2εn} + \frac{|S_{\mathrm{out}}|}{γn})$。
- 适用于隐私保护场景中存在少量异常数据的二分类任务，尤其适合对鲁棒性要求高的应用。
本文研究二分类场景下的差分隐私经验风险最小化（DP-ERM）问题。提出一种高效的 $(\varepsilon,δ)$-差分隐私算法，其经验0-1损失上界为 $\tilde{O}\left(\frac{1}{γ^2\varepsilon n} + \frac{|S_{\mathrm{out}}|}{γn}\right)$，其中 $n$ 为样本数，$S_{\mathrm{out}}$ 为可移除的任意数据子集，$γ$ 为移除后剩余数据的线性间隔。$\tilde{O}(\cdot)$ 仅隐藏对数项。在无假设情形下，当异常点数量较小时，该结果优于现有方法。算法具有高度自适应性，无需预先知道 $γ$ 或 $S_{\mathrm{out}}$。此外，还推导了先进私有超参数调优算法的效用界。
原文摘要 · Abstract (English)
This paper studies the problem of differentially private empirical risk minimization (DP-ERM) for binary linear classification. We obtain an efficient $(\varepsilon,δ)$-DP algorithm with an empirical zero-one risk bound of $\tilde{O}\left(\frac{1}{γ^2\varepsilon n} + \frac{|S_{\mathrm{out}}|}{γn}\right)$ where $n$ is the number of data points, $S_{\mathrm{out}}$ is an arbitrary subset of data one can remove and $γ$ is the margin of linear separation of the remaining data points (after $S_{\mathrm{out}}$ is removed). Here, $\tilde{O}(\cdot)$ hides only logarithmic terms. In the agnostic case, we improve the existing results when the number of outliers is small. Our algorithm is highly adaptive because it does not require knowing the margin parameter $γ$ or outlier subset $S_{\mathrm{out}}$. We also derive a utility bound for the advanced private hyperparameter tuning algorithm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。