提出精确的谱估计方法，实现低维信号子空间的最优弱恢复。
Spectral Estimators for Multi-Index Models: Precise Asymptotics and Optimal Weak Recovery
- 基于谱方法分析信号子空间的渐近性能，给出特征值与特征向量的精准刻画。
- 发现样本量增长时存在相变现象，特征值脱离谱分布主干后可恢复信号方向。
- 揭示数据预处理优化路径，找到实现弱恢复所需最少样本的最优谱估计器。
多指标模型是研究具有低维结构函数可学习性的流行框架，且因其与神经网络的关联而受到近期广泛关注。本文聚焦于通过谱估计器恢复信号所张成的子空间——这类方法在实践中常作为迭代算法的初始化。主要技术贡献在于当样本量与输入维度同比例增长且待恢复空间维度 $p$ 固定时，对谱方法性能进行精确渐近分析。具体地，我们定位了谱矩阵的前 $p$ 个最大特征值，并建立了对应特征向量与信号子空间基之间的重叠关系。分析揭示了一种相变现象：随着样本复杂度增加，特征值从谱分布主干中分离，此时特征向量开始恢复目标子空间的方向。所提出的精确刻画使数据预处理得以优化，从而识别出实现弱恢复所需样本量最小的谱估计器。
原文摘要 · Abstract (English)
Multi-index models provide a popular framework to investigate the learnability of functions with low-dimensional structure and, also due to their connections with neural networks, they have been object of recent intensive study. In this paper, we focus on recovering the subspace spanned by the signals via spectral estimators -- a family of methods routinely used in practice, often as a warm-start for iterative algorithms. Our main technical contribution is a precise asymptotic characterization of the performance of spectral methods, when sample size and input dimension grow proportionally and the dimension $p$ of the space to recover is fixed. Specifically, we locate the top-$p$ eigenvalues of the spectral matrix and establish the overlaps between the corresponding eigenvectors (which give the spectral estimators) and a basis of the signal subspace. Our analysis unveils a phase transition phenomenon in which, as the sample complexity grows, eigenvalues escape from the bulk of the spectrum and, when that happens, eigenvectors recover directions of the desired subspace. The precise characterization we put forward enables the optimization of the data preprocessing, thus allowing to identify the spectral estimator that requires the minimal sample size for weak recovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。