arXiv:2609.07432cs.LGcs.AI2026-09

提出两种新方法,高效压缩核学习数据集并提升训练速度。

Revisiting Thinning Methods for Kernel Learning Problems

  • 反向核聚类法逐点移除数据,加速子集生成。
  • 可任意设定子集大小,突破传统减半限制。
  • 适合对训练效率或预测性能有不同需求的场景。

核方法因理论保证和实际表现优异而广泛应用,但其高计算成本限制了在大规模数据上的应用。为此,已有方法利用最大均值差异构建能保留完整数据集特性的代表性子集。本文提出反向核聚类算法,通过迭代移除数据点,在子集规模小于原数据一半的实际场景中,实现与当前最优方法相当的效果,并显著加快采样过程。同时,我们改进核稀释法,使其可构造任意大小的子集,不再局限于连续减半。我们在高斯过程与核支持向量机等核心核学习任务上进行了全面实验。结果表明,反向核聚类在训练时间效率上表现优异且稳定;而柔性核稀释法在预测性能上常取得最佳效果,尤其在中等压缩比下优势明显,凸显引入监督信息的价值。内存消耗方面,柔性核稀释法也具竞争力,而反向核聚类则更适合以计算效率为首要目标的场景。总体而言,无单一方法在所有情况下占优,应根据预测性能、训练成本与内存需求权衡选择策略。

原文摘要 · Abstract (English)

Kernel methods are widely used because of their strong theoretical guarantees and empirical performance. However, their high computational cost limits their applicability to large-scale datasets. To address this shortcoming, several approaches use Maximum Mean Discrepancy to construct representative subsets that preserve the properties of the full dataset in a Reproducing Kernel Hilbert Space. We introduce Backward Kernel Herding, an algorithm that addresses this problem by iteratively removing points from the dataset, achieving results comparable to current state-of-the-art approaches while accelerating the subsampling process in realistic scenarios where the reduced size is less than half of the dataset. Moreover, we overcome a limitation of Kernel Thinning by proposing an extension that enables the construction of subsets of arbitrary size rather that restricting to successive halvings. Finally, we conduct an extensive experimental comparison focusing on the most relevant kernel learning procedures: Gaussian Processes and Kernel Support Vector Machines. The results show that Backward Kernel Herding consistently achieves competitive performance with the most favorable training-time efficiency, while the proposed Flexible Kernel Thinning frequently achieves the best predictive performance. These gains become especially pronounced for moderate compression ratios, highlighting the benefits of incorporating supervised information into the thinning process. In terms of memory consumption, Flexible Kernel Thinning is also competitive, whereas Backward Kernel Herding remains an alternative when computational efficiency is the primary objective. Overall, no single method dominates across all scenarios, underscoring the importance of selecting the reduction strategy according to the desired trade-off between predictive performance, training cost, and memory requirements.

核方法数据压缩高效学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。