提出新凸代理函数，提升非线性降维中低维特征空间的逼近精度。
Surrogate to Poincaré inequalities on manifolds for dimension reduction in nonlinear feature spaces
- 用凸代理替代原复杂损失函数，简化优化过程。
- 小样本下表现更优，尤其当降维至一维时误差显著降低。
- 适用于多项式函数和多种数据分布，适合高维数据降维场景。
我们旨在通过两阶段构造法近似一个连续可微函数 $u:bR^d o bR$，即表示为 $figcirc g$，其中 $g:bR^d o bR^m$（$m eq d$）和 $f:bR^m o bR$。固定 $g$ 后，利用经典回归方法构建 $f$，需评估 $u$。已有工作通过最小化基于流形上 Poincaré 不等式的损失 $J(g)$ 来构建非线性 $g$，该过程依赖 $u$ 的梯度评估，但优化困难。本文引入新的凸代理函数以替代 $J$，并借助集中不等式，为包括多项式在内的函数类及广泛输入概率测度提供次优性结果。在不同基准测试中，验证了在各种训练样本规模下的性能，表明所提方法优于标准迭代优化方式，在小样本和 $m=1$ 时常获得更低逼近误差。
原文摘要 · Abstract (English)
We aim to approximate a continuously differentiable function $u:\mathbb{R}^d \rightarrow \mathbb{R}$ by a composition of functions $f\circ g$ where $g:\mathbb{R}^d \rightarrow \mathbb{R}^m$, $m\leq d$, and $f : \mathbb{R}^m \rightarrow \mathbb{R}$ are built in a two stage procedure. For a fixed $g$, we build $f$ using classical regression methods, involving evaluations of $u$. Recent works proposed to build a nonlinear $g$ by minimizing a loss function $\mathcal{J}(g)$ derived from Poincaré inequalities on manifolds, involving evaluations of the gradient of $u$. A problem is that minimizing $\mathcal{J}$ may be a challenging task. Hence in this work, we introduce new convex surrogates to $\mathcal{J}$. Leveraging concentration inequalities, we provide suboptimality results for a class of functions $g$, including polynomials, and a wide class of input probability measures. We investigate performances on different benchmarks for various training sample sizes. We show that our approach outperforms standard iterative methods for minimizing the training Poincaré inequality based loss, often resulting in better approximation errors, especially for small training sets and $m=1$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。