提出超核岭回归，让模型自动适应高维组合结构。
Learning Multi-Index Models with Hyper-Kernel Ridge Regression
- 结合神经网络与核方法，设计超核岭回归新算法
- 样本复杂度分析表明可突破维度诅咒限制
- 适合研究深度学习理论或高维建模的学者
深度神经网络在高维问题中表现优异，超越了受维度诅咒影响的核方法。然而其成功背后的理论基础仍不清晰。本文认为学习任务的组合结构是决定深度网络优势的关键。为此，我们研究一种简单组合模型——多指标模型（MIM），提出并分析超核岭回归（HKRR），该方法融合神经网络与核方法。主要贡献是给出了样本复杂度结果，证明HKRR能自适应学习MIM，克服维度诅咒。此外，利用估计器的核特性，发展了专用优化方法。对比了交替最小化与交替梯度法，理论与数值结果均支持该方法的有效性。
原文摘要 · Abstract (English)
Deep neural networks excel in high-dimensional problems, outperforming models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical foundations of this success remain poorly understood. We follow the idea that the compositional structure of the learning task is the key factor determining when deep networks outperform other approaches. Taking a step towards formalizing this idea, we consider a simple compositional model, namely the multi-index model (MIM). In this context, we introduce and study hyper-kernel ridge regression (HKRR), an approach blending neural networks and kernel methods. Our main contribution is a sample complexity result demonstrating that HKRR can adaptively learn MIM, overcoming the curse of dimensionality. Further, we exploit the kernel nature of the estimator to develop ad hoc optimization approaches. Indeed, we contrast alternating minimization and alternating gradient methods both theoretically and numerically. These numerical results complement and reinforce our theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。