根据局部曲率自适应调整邻居数,提升小样本分类性能
Adaptive $k$-nearest neighbor classifier based on the local estimation of the shape operator
- 依据局部形状算子估计曲率,动态调整邻域大小
- 在多个真实数据集上平衡准确率优于传统k-NN和其它自适应方法
- 尤其在训练样本少时表现更优,适合小样本学习场景
k-最近邻(k-NN)是非参数分类中最常用的方法之一。然而,邻居数k的设定存在关键限制,直接影响偏差-方差权衡、决策边界平滑性、抗噪能力及类别不平衡处理。本文提出一种新的自适应k-NN(kK-NN)算法,通过估计样本点的局部曲率来动态确定邻域大小:低曲率区域采用较大邻域(切空间近似良好),高曲率区域采用较小邻域(切空间近似较差)。利用局部协方差矩阵与局部海塞矩阵估算局部高斯曲率。大量真实数据集上的实验表明,新方法在平衡准确率上优于标准k-NN及其他自适应k-NN算法,尤其在训练样本有限时表现突出,说明kK-NN能在较少数据下学习更具判别性的分类函数。
原文摘要 · Abstract (English)
The $k$-nearest neighbor ($k$-NN) algorithm is one of the most popular methods for nonparametric classification. However, a relevant limitation concerns the definition of the number of neighbors $k$. This parameter exerts a direct impact on several properties of the classifier, such as the bias-variance tradeoff, smoothness of decision boundaries, robustness to noise, and class imbalance handling. In the present paper, we introduce a new adaptive $k$-nearest neighbours ($kK$-NN) algorithm that explores the local curvature at a sample to adaptively defining the neighborhood size. The rationale is that points with low curvature could have larger neighborhoods (locally, the tangent space approximates well the underlying data shape), whereas points with high curvature could have smaller neighborhoods (locally, the tangent space is a loose approximation). We estimate the local Gaussian curvature by computing an approximation to the local shape operator in terms of the local covariance matrix as well as the local Hessian matrix. Results on many real-world datasets indicate that the new $kK$-NN algorithm yields superior balanced accuracy compared to the established $k$-NN method and also another adaptive $k$-NN algorithm. This is particularly evident when the number of samples in the training data is limited, suggesting that the $kK$-NN is capable of learning more discriminant functions with less data considering many relevant cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。