arXiv:2505.21336cs.LGmath.OC2025-05被引 1

同时学习高维数据的投影方向和非线性函数,突破传统方法局限。

Joint Learning in the Gaussian Single Index Model

  • 通过交替梯度流联合优化方向与非线性函数,解决非凸问题。
  • 即使初始方向与目标负相关,仍能收敛,速率由函数高斯正则性决定。
  • 基于再生核希尔伯特空间实现高效灵活的函数估计,适合高维建模。

我们研究在高维高斯模型中联合学习一维投影方向与一元函数的问题。具体地,考虑形如 $f(x)=φ^ ext{⋆}(\< w^ ext{⋆}, x angle)$ 的预测器,其中方向 $w^ ext{⋆} \in \mathcal{S}_{d-1}$ 和函数 $φ^ ext{⋆}: \mathbb{R} \to \mathbb{R}$ 均从高斯数据中学习。该设定刻画了表示学习与非线性回归交汇处的基本非凸问题。我们分析了一种自然交替方案的梯度流动力学,证明其收敛性,且收敛速率受反映 $φ^ ext{⋆}$ 高斯正则性的信息指数控制。令人惊讶的是,即使初始方向与目标负相关,收敛依然发生。在实践层面,我们展示了可通过适配问题结构的再生核希尔伯特空间(RKHS)有效实现联合学习,实现对一元函数的高效且灵活估计。结果为高维场景中低维结构学习提供了理论洞察与实用方法。

原文摘要 · Abstract (English)

We consider the problem of jointly learning a one-dimensional projection and a univariate function in high-dimensional Gaussian models. Specifically, we study predictors of the form $f(x)=φ^\star(\langle w^\star, x \rangle)$, where both the direction $w^\star \in \mathcal{S}_{d-1}$, the sphere of $\mathbb{R}^d$, and the function $φ^\star: \mathbb{R} \to \mathbb{R}$ are learned from Gaussian data. This setting captures a fundamental non-convex problem at the intersection of representation learning and nonlinear regression. We analyze the gradient flow dynamics of a natural alternating scheme and prove convergence, with a rate controlled by the information exponent reflecting the \textit{Gaussian regularity} of the function $φ^\star$. Strikingly, our analysis shows that convergence still occurs even when the initial direction is negatively correlated with the target. On the practical side, we demonstrate that such joint learning can be effectively implemented using a Reproducing Kernel Hilbert Space (RKHS) adapted to the structure of the problem, enabling efficient and flexible estimation of the univariate function. Our results offer both theoretical insight and practical methodology for learning low-dimensional structure in high-dimensional settings.

高维统计非线性回归表示学习梯度流

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。