提出可学习特征的组合核模型，揭示非线性特征恢复机制。
A Compositional Kernel Model for Feature Learning
- 用坐标加权输入的组合核回归，建模特征学习过程。
- 在高斯噪声下，$\ ext{Laplace}$核能恢复非线性特征，高斯核仅恢复线性特征。
- 适用于研究神经网络中变量选择与特征提取的理论分析。
我们研究了一种组合型核岭回归，其中预测器作用于输入的逐坐标重加权。该模型可表述为变分问题，为组合架构中的特征学习提供了可处理的分析框架。从变量选择视角出发，我们证明了相关变量可被恢复，而噪声变量会被剔除。当噪声变量服从高斯分布时，全局极小值点与驻点均会丢弃噪声坐标。核心发现是：$\ell_1$ 类核（如拉普拉斯核）在驻点处能恢复对非线性效应有贡献的特征，而高斯核仅能恢复线性特征。
原文摘要 · Abstract (English)
We study a compositional variant of kernel ridge regression in which the predictor is applied to a coordinate-wise reweighting of the inputs. Formulated as a variational problem, this model provides a tractable setting for studying feature learning in compositional architectures. From the perspective of variable selection, we show how relevant variables are recovered while noise variables are eliminated. We prove that both global minimizers and stationary points discard noise coordinates when the noise variables are Gaussian distributed. A central finding is that $\ell_1$-type kernels, such as the Laplace kernel, succeed in recovering features contributing to nonlinear effects at stationary points, whereas Gaussian kernels recover only linear ones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。