arXiv:2502.11413cs.LGstat.ML2025-02ICML被引 4

三类以上线性分类在随机噪声下难以高效学习,理论证明存在超多项式复杂度瓶颈。

Statistical Query Hardness of Multiclass Linear Classification with Random Classification Noise

  • 基于统计查询框架,揭示多类线性分类在随机噪声下的计算难题
  • 即使三类且噪声分离度恒定,最优误差学习仍需超多项式时间
  • 结果适用于高维、小分离度场景,对算法设计有根本性限制

我们研究在无分布PAC模型中带有随机分类噪声(RCN)的多类线性分类(MLC)任务。学习者接收来自未知分布的样本 $(x, y)$,其中标签由一个多类线性分类器受噪声影响生成:标签 $y$ 从 $i$ 被翻转为 $j$ 的概率为已知噪声矩阵 $H_{ij}$,其非负分离度定义为 $σ: = \min_{i \neq j} H_{ii}-H_{ij}$。目标是构造一个0-1误差接近最小的假设。对于两类情形,已有多项式时间算法达到最优误差。但对三类及以上情况,复杂性尚不明确。本文主要贡献在于:当类别数≥3时,该问题的统计查询(SQ)复杂度发生质变——即使三类且分离度恒定,任何实现最优误差的SQ算法仍需超多项式时间;当类别数更多、分离度更小时,即使仅要求常数倍近似或超越平凡猜测,也存在超多项式下界。

原文摘要 · Abstract (English)

We study the task of Multiclass Linear Classification (MLC) in the distribution-free PAC model with Random Classification Noise (RCN). Specifically, the learner is given a set of labeled examples $(x, y)$, where $x$ is drawn from an unknown distribution on $R^d$ and the labels are generated by a multiclass linear classifier corrupted with RCN. That is, the label $y$ is flipped from $i$ to $j$ with probability $H_{ij}$ according to a known noise matrix $H$ with non-negative separation $σ: = \min_{i \neq j} H_{ii}-H_{ij}$. The goal is to compute a hypothesis with small 0-1 error. For the special case of two labels, prior work has given polynomial-time algorithms achieving the optimal error. Surprisingly, little is known about the complexity of this task even for three labels. As our main contribution, we show that the complexity of MLC with RCN becomes drastically different in the presence of three or more labels. Specifically, we prove super-polynomial Statistical Query (SQ) lower bounds for this problem. In more detail, even for three labels and constant separation, we give a super-polynomial lower bound on the complexity of any SQ algorithm achieving optimal error. For a larger number of labels and smaller separation, we show a super-polynomial SQ lower bound even for the weaker goal of achieving any constant factor approximation to the optimal loss or even beating the trivial hypothesis.

机器学习统计查询多分类复杂性理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。