提出新指数，高效学习低维投影的高斯多指标模型。
The Generative Leap: Sharp Sample Complexity for Efficiently Learning Gaussian Multi-Index Models
- 引入生成跃迁指数k⋆，扩展单指标情形的生成指数。
- 证明样本复杂度n=Θ(d^{1∨k/2})在低阶多项式框架下不可更优。
- 设计无需先验知识的谱U统计量序列估计法，适用于深度网络等结构。
本文研究通用高斯多指标模型，其中标签仅通过输入在低维r=O_d(1)子空间上的投影决定。我们引入生成跃迁指数k⋆，作为[Damian et al., '24]中生成指数在多指标情形的自然推广。首先证明，在低阶多项式框架下，样本复杂度n=Θ(d^{1∨k/2})是必需的。随后，我们通过基于适当赫尔米特张量的谱U统计量，提出一种无需先验知识的广义序贯估计方法，证明该复杂度亦充分。进一步计算了若干典型情形的生成跃迁指数，包括分段线性函数（带偏置的深层ReLU网络）及具有r维第一隐层的通用深层神经网络。
原文摘要 · Abstract (English)
In this work we consider generic Gaussian Multi-index models, in which the labels only depend on the (Gaussian) $d$-dimensional inputs through their projection onto a low-dimensional $r = O_d(1)$ subspace, and we study efficient agnostic estimation procedures for this hidden subspace. We introduce the \emph{generative leap} exponent $k^\star$, a natural extension of the generative exponent from [Damian et al.'24] to the multi-index setting. We first show that a sample complexity of $n=Θ(d^{1 \vee \k/2})$ is necessary in the class of algorithms captured by the Low-Degree-Polynomial framework. We then establish that this sample complexity is also sufficient, by giving an agnostic sequential estimation procedure (that is, requiring no prior knowledge of the multi-index model) based on a spectral U-statistic over appropriate Hermite tensors. We further compute the generative leap exponent for several examples including piecewise linear functions (deep ReLU networks with bias), and general deep neural networks (with $r$-dimensional first hidden layer).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。