arXiv:2502.15900cs.LGstat.ML2025-02被引 161

解释近邻方法为何在预测中有效,从理论到实践全覆盖。

Explaining the Success of Nearest Neighbor Methods in Prediction

论文配图:Explaining the Success of Nearest Neighbor Methods in Prediction
图 1 · 摘自论文原文
  • 基于非渐近统计保证,分析近邻预测的理论基础
  • 在时间序列、推荐系统和医学图像中验证近邻方法有效性
  • 揭示聚类结构是近邻成功的关键,适合数据有局部相似性的场景

许多现代预测方法利用近邻搜索找到与测试样本最相似的历史训练样本,这一思想可追溯至11世纪,在文本领域已有悠久历史。本文旨在解释该方法的成功原因,涵盖理论与实践两方面:理论上,提供近邻回归与分类的非渐近统计保证,明确在何种训练数据量和算法参数下能达到指定误差容忍度;在一般度量空间中,预测成功依赖于被估计函数的平滑性(回归)或决策边界附近的低概率(分类)。实践中,整理了关键的近似近邻搜索方法,使大规模数据下的近邻分析成为可能。此外,讨论了学习距离以优化近邻方法的进展,包括随机决策树与集成方法如何隐式学习近邻结构,以及近期在众包与图论模型中的应用。最后通过三个案例研究——时间序列预测、动态用户推荐、医学图像器官分割——展示聚类结构如何促进近邻方法的有效性。

原文摘要 · Abstract (English)

Many modern methods for prediction leverage nearest neighbor search to find past training examples most similar to a test example, an idea that dates back in text to at least the 11th century and has stood the test of time. This monograph aims to explain the success of these methods, both in theory, for which we cover foundational nonasymptotic statistical guarantees on nearest-neighbor-based regression and classification, and in practice, for which we gather prominent methods for approximate nearest neighbor search that have been essential to scaling prediction systems reliant on nearest neighbor analysis to handle massive datasets. Furthermore, we discuss connections to learning distances for use with nearest neighbor methods, including how random decision trees and ensemble methods learn nearest neighbor structure, as well as recent developments in crowdsourcing and graphons. In terms of theory, our focus is on nonasymptotic statistical guarantees, which we state in the form of how many training data and what algorithm parameters ensure that a nearest neighbor prediction method achieves a user-specified error tolerance. We begin with the most general of such results for nearest neighbor and related kernel regression and classification in general metric spaces. In such settings in which we assume very little structure, what enables successful prediction is smoothness in the function being estimated for regression, and a low probability of landing near the decision boundary for classification. In practice, these conditions could be difficult to verify for a real dataset. We then cover recent guarantees on nearest neighbor prediction in the three case studies of time series forecasting, recommending products to people over time, and delineating human organs in medical images by looking at image patches. In these case studies, clustering structure enables successful prediction.

近邻方法统计理论预测模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。