提出新算法在可分解分布下高效学习两个半空间交集
Learning Intersections of Two Margin Halfspaces under Factorizable Distributions
- 基于广义统计查询,结合矩张量与主成分分析
- 实现多项式时间 $poly(d,1/γ)$,突破原有准多项式瓶颈
- 适用于可分解分布,适合研究学习复杂度的理论学者
学习半空间交集是计算学习理论中的核心问题。即使仅针对两个半空间,能否在关于数据点边际 $γ$ 和维度 $d$ 的多项式时间内完成学习,仍是未解难题。现有最佳算法需准多项式时间 $d^{O(\log(1/γ))}$,且已证明依赖相关性统计查询(CSQ)的算法无法避免此复杂度。本文提出一种新算法,可突破该硬性障碍。该方法适用于满足自然因子分解假设的一类广泛分布,介于特定分布与无分布假设之间,显著扩展了已知可解情形。在这些分布下,基于 CSQ 的方法仍需准多项式时间,而我们的算法通过更一般的统计查询(SQ)实现 $poly(d,1/γ)$ 时间,揭示了此类简单可实现 PAC 学习问题中 CSQ 与 SQ 的强分离性。结果基于新颖对偶框架的严格分析,刻画了边缘分布诱导的矩张量结构。据此,我们设计了高效算法:结合改进的 Jennrich 算法、随机投影上的主成分分析及基于梯度下降的非凸优化框架。
原文摘要 · Abstract (English)
Learning intersections of halfspaces is a central problem in Computational Learning Theory. Even for just two halfspaces, it remains a major open question whether learning is possible in polynomial time with respect to the margin $γ$ of the data points and their dimensionality $d$. The best-known algorithms run in quasi-polynomial time $d^{O(\log(1/γ))}$, and it has been shown that this complexity is unavoidable for any algorithm relying solely on correlational statistical queries (CSQ). In this work, we introduce a novel algorithm that provably circumvents the CSQ hardness barrier. Our approach applies to a broad class of distributions satisfying a natural, previously studied, factorizability assumption. Factorizable distributions lie between distribution-specific and distribution-free settings, and significantly extend previously known tractable cases. Under these distributions, we show that CSQ-based methods still require quasipolynomial time even for weakly learning, whereas our algorithm achieves $poly(d,1/γ)$ time by leveraging more general statistical queries (SQ), establishing a strong separation between CSQ and SQ for this simple realizable PAC learning problem. Our result is grounded in a rigorous analysis utilizing a novel duality framework that characterizes the moment tensor structure induced by the marginal distributions. Building on these structural insights, we propose new, efficient learning algorithms. These algorithms combine a refined variant of Jennrich's Algorithm with PCA over random projections of the moment tensor, along with a gradient-descent-based non-convex optimization framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。