提出自适应加权kNN图,显著提升图拉普拉斯收敛速度。
Improved convergence rate of kNN graph Laplacians: differentiable self-tuned affinity
- 基于kNN距离自适应调整核带宽,构造可微分的加权图。
- 在流形数据下实现O(N^{-2/(d+6)})的收敛率,优于传统方法。
- 理论严谨且可验证,适合流形学习与图分析研究者。
在基于图的数据分析中,k最近邻(kNN)图因其对局部数据密度的自适应性而广泛应用。通过引入加权边,核化图亲和度提供了一种更通用的kNN图形式,其中kNN距离被用于自适应设定核带宽。本文考虑一类广义kNN图,其图亲和度为 $W_{ij} = ε^{-d/2} k_0 ( \| x_i - x_j \|^2 / εϕ( \hat ρ(x_i), \hat ρ(x_j) )^2 ) $,其中 $\hatρ(x)$ 为点 $x$ 处的(归一化)kNN距离,$ϕ$ 为对称双变量函数,$k_0$ 为定义在 $[0,∞)$ 上的非负函数。在流形数据设定下,当 $N$ 个独立同分布样本 $x_i$ 从嵌入高维欧氏空间的 $d$ 维未知流形上的密度 $p$ 中抽取时,我们证明了kNN图拉普拉斯算子在点态上收敛到依赖于 $p$ 的极限流形算子,收敛速率为 $O(N^{-2/(d+6)})$,至多包含对数因子,前提是 $k_0$ 与 $ϕ$ 满足 $C^3$ 正则性及其他技术条件。该结果在 $ε\sim N^{-2/(d+6)}$ 且 $k\sim N^{6/(d+6)}$ 时成立,两者均处于平衡理论偏差与方差误差的最优阶。改进的收敛速率源于对kNN估计器的精细分析,该分析本身可能具有独立价值。我们在模拟数据上通过数值实验验证了理论结果。
原文摘要 · Abstract (English)
In graph-based data analysis, $k$-nearest neighbor ($k$NN) graphs are widely used due to their adaptivity to local data densities. Allowing weighted edges in the graph, the kernelized graph affinity provides a more general type of $k$NN graph where the $k$NN distance is used to set the kernel bandwidth adaptively. In this work, we consider a general class of $k$NN graph where the graph affinity is $W_{ij} = ε^{-d/2} k_0 ( \| x_i - x_j \|^2 / εϕ( \hat ρ(x_i), \hat ρ(x_j) )^2 ) $, with $\hatρ(x)$ being the (rescaled) $k$NN distance at the point $x$, $ϕ$ a symmetric bi-variate function, and $k_0$ a non-negative function on $[0,\infty)$. Under the manifold data setting, where $N$ i.i.d. samples $x_i$ are drawn from a density $p$ on a $d$-dimensional unknown manifold embedded in a high dimensional Euclidean space, we prove the operator pointwise convergence of the $k$NN graph Laplacian to the limiting manifold operator (depending on $p$) at the rate of $O(N^{-2/(d+6)})$, up to a log factor, when $k_0$ and $ϕ$ have $C^3$ regularity and satisfy other technical conditions. This is obtained when $ε\sim N^{-2/(d+6)}$ and $k \sim N^{6/(d+6)}$, both at the optimal order to balance the theoretical bias and variance errors. Our improved convergence rate is based on a refined analysis of the $k$NN estimator, which can be of independent interest. We validate our theory by numerical experiments on simulated data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。