研究分布鲁棒学习的样本复杂度,揭示了鲁棒性对学习效率的影响机制。
The Sample Complexity of Distributionally Robust PAC Learning under Cressie--Read Divergences
- 基于Cressie-Read散度构建分布鲁棒学习框架,分析样本复杂度边界。
- 在真实与非真实情形下,分别获得紧致的样本复杂度上界,且与经验风险最小化性能匹配。
- 首次统一处理所有k>1阶的Cressie-Read散度,填补理论空白,适合理论学习研究者。
研究在0-1损失下,以Cressie--Read散度(阶数k>1,半径ρ≥0)约束数据分布扰动的分布鲁棒PAC学习。对于VC维为d的假设类,本文建立了实可学习与广义学习情形下紧致的样本复杂度上界(常数与对数因子内最优);普通经验风险最小化在对数因子意义下达到这些率。目标精度ε∈(0,1),置信度δ∈(0,1)时,其量级分别为:max{1/ε, ρ^{1/(k−1)} / ε^{k⋆}}·(d+log δ^{-1}) 和 max{1/ε², ρ^{1/(k−1)} / ε^{k⋆∨2}}·(d+log δ^{-1}),其中k⋆=k/(k−1)。当固定ρ>0时,鲁棒性将实可学习情形的ε依赖从ε^{-1}变为ε^{-k⋆}(ε↓0)。在广义情形中,当1<k<2时,ε依赖由ε^{-2}变为ε^{-k⋆};而当k≥2时,指数仍保持经典2,但存在非平凡的ρ依赖。通过已知的标量化技巧将鲁棒0-1风险转化为普通分类误差,本分析揭示了统计估计与鲁棒性放大的敏感尺度交互,清晰解释了广义率中的过渡现象。将此前仅限χ²散度的情况扩展至任意k>1阶的Cressie--Read散度,弥合上下界差距,并在ρ→0时正确恢复标准PAC学习率,优于以往无法在极限下插值的边界。
原文摘要 · Abstract (English)
We study distributionally robust PAC learning for the $0$--$1$-loss, where adversarial perturbations of the data distribution are constrained by a Cressie--Read divergence of order $k>1$ and radius $ρ\geq 0$. For hypothesis classes with VC dimension $d$, we establish realizable and agnostic sample-complexity bounds tight up to constant and logarithmic factors, respectively; ordinary empirical risk minimization attains both rates up to logarithmic factors. For target accuracy $\varepsilon\in(0,1)$ and confidence $δ\in(0,1)$, their respective orders are \[ \max\!\left\{\frac{1}{\varepsilon}, \frac{ρ^{\frac 1{k-1}}}{\varepsilon^{k_\star}} \right\}\cdot(d+\log δ^{-1}) \qquad\text{and}\qquad \max\!\left\{\frac{1}{\varepsilon^2}, \frac{ρ^{\frac1{k-1}}}{\varepsilon^{k_\star\vee 2}} \right\}\cdot(d+\log δ^{-1}), \] where $k_\star={k}/{(k-1)}$. For every fixed $ρ>0$, robustness changes the realizable $\varepsilon$-dependence from $\varepsilon^{-1}$ to $\varepsilon^{-k_\star}$ as $\varepsilon\downarrow0$. In the agnostic case, for $1<k<2$, robustness changes the $\varepsilon$-dependence from $\varepsilon^{-2}$ to $\varepsilon^{-k_\star}$, whereas for $k\geq2$ the exponent remains the classical $2$, with nontrivial $ρ$-dependence. Building on the known scalar reduction of robust $0$--$1$ risk to ordinary classification error, our analysis reveals a scale-sensitive interaction between the statistical estimation of classification error and its amplification by robustness, sharply explaining the transition in the agnostic rate. We extend the previously studied $χ^2$-divergence case to every Cressie--Read order $k>1$, close its upper--lower gaps, and recover standard PAC learning rates as $ρ\to0$, unlike previous bounds that fail to interpolate correctly in this limit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。