提出局部EGOP方法,实现高维函数的连续索引学习。
Local EGOP for Continuous Index Learning
- 基于期望梯度外积构造局部子空间度量,自适应捕捉函数变化方向。
- 在噪声流形假设下,实现与函数内在维度匹配的学习速率。
- 相比两层神经网络,回归精度显著提升,适用于高维数据建模。
我们提出连续索引学习的新设定:函数在每一点仅沿少数方向变化。为高效估计,学习算法需在每个点x附近自适应地捕捉函数局部变异性所对应的子空间。我们将该任务建模为带噪声的流形上的核适应问题,提出局部EGOP学习——一种递归算法,利用期望梯度外积(EGOP)二次型作为度量和目标分布的逆协方差。我们证明,在监督噪声流形假设下,该算法可实现与函数内在维度一致的学习速率,且对任意高维噪声均成立。实验上,我们将算法与深度学习的特征学习能力对比,并在连续单指数设定中验证其回归性能优于两层神经网络。
原文摘要 · Abstract (English)
We introduce the setting of continuous index learning, in which a function of many variables varies only along a small number of directions at each point. For efficient estimation, it is beneficial for a learning algorithm to adapt, near each point $x$, to the subspace that captures the local variability of the function $f$. We pose this task as kernel adaptation along a manifold with noise, and introduce Local EGOP learning, a recursive algorithm that utilizes the Expected Gradient Outer Product (EGOP) quadratic form as both a metric and inverse-covariance of our target distribution. We prove that Local EGOP learning adapts to the regularity of the function of interest, showing that under a supervised noisy manifold hypothesis, intrinsic dimensional learning rates are achieved for arbitrarily high-dimensional noise. Empirically, we compare our algorithm to the feature learning capabilities of deep learning. Additionally, we demonstrate improved regression quality compared to two-layer neural networks in the continuous single-index setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。