提出首个带误差保证的半空间选择分类算法,适用于高斯分布数据。
Distribution-Specific Agnostic Conditional Classification With Halfspaces
- 用齐次半空间作为选择规则,设计可学习的条件分类器。
- 在高斯分布下实现误差为√opt*的近似最优解,首次给出理论保证。
- 证明该问题在密码学假设下计算困难,适合关注理论边界的研究者。
我们研究了在无认知(agnostic)设定下的‘选择性’或‘条件’分类问题。传统分类聚焦于建模特征与类别间的关系以覆盖大部分数据,而条件分类仅针对由某种选择规则定义的数据子集进行建模。现有工作大多在可实现设定下求解,或无法保证误差相对于最优解有界。本文考虑使用稀疏线性分类器对由半空间定义的子集进行条件分类,在高斯特征分布下给出正负结果。正面结果方面,提出首个针对齐次半空间选择器的PAC学习算法,其误差保证为√opt*,其中opt是给定分类器类与齐次半空间中的最小条件分类误差。负面结果方面,在密码学假设下,即使在高斯分布下,逼近条件分类损失的小加性误差也是计算上困难的,并证明其难度至少等同于逼近无认知分类的加法和乘法形式。
原文摘要 · Abstract (English)
We study ``selective'' or ``conditional'' classification problems under an agnostic setting. Classification tasks commonly focus on modeling the relationship between features and categories that captures the vast majority of data. In contrast to common machine learning frameworks, conditional classification intends to model such relationships only on a subset of the data defined by some selection rule. Most work on conditional classification either solves the problem in a realizable setting or does not guarantee the error is bounded compared to an optimal solution. In this work, we consider selective/conditional classification by sparse linear classifiers for subsets defined by halfspaces, and give both positive as well as negative results for Gaussian feature distributions. On the positive side, we present the first PAC-learning algorithm for homogeneous halfspace selectors with error guarantee $\bigO*{\sqrt{\mathrm{opt}}}$, where $\mathrm{opt}$ is the smallest conditional classification error over the given class of classifiers and homogeneous halfspaces. On the negative side, we find that, under cryptographic assumptions, approximating the conditional classification loss within a small additive error is computationally hard even under Gaussian distribution. We prove that approximating conditional classification is at least as hard as approximating agnostic classification in both additive and multiplicative form.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。