arXiv:2507.22299cs.LG2025-07被引 2

对比聚类交叉验证方法,发现分层法在不平衡数据上更可靠。

Comparing Cluster-Based Cross-Validation Strategies for Machine Learning Model Evaluation

  • 用聚类算法分组数据再交叉验证,避免样本分布偏差。
  • 小批量K均值+分层在平衡数据上偏差小、方差低。
  • 不平衡数据仍推荐传统分层交叉验证,效果更稳。

交叉验证是机器学习中评估模型性能的关键方法,但传统划分可能无法代表原始数据的多样性,导致性能估计偏差。本文通过实验比较多种基于聚类的交叉验证策略,分析不同聚类算法的表现,并提出一种结合小批量K均值与类别分层的新方法。在20个数据集(含平衡与不平衡)上,使用4种监督学习算法进行测试,评估其偏差、方差和计算成本。结果表明:在平衡数据上,新方法在偏差和方差方面优于其他策略,但计算成本未显著降低;在不平衡数据上,传统分层交叉验证始终表现最佳,具有更低偏差、方差和计算开销,是此类场景下的稳妥选择。不同聚类算法间无明显优劣。本研究深化了对聚类交叉验证的理解,重申了分层交叉验证的有效性,并为具有聚类特性的数据提供了更鲁棒的评估思路。

原文摘要 · Abstract (English)

Cross-validation plays a fundamental role in Machine Learning, enabling robust evaluation of model performance and preventing overestimation on training and validation data. However, one of its drawbacks is the potential to create data subsets (folds) that do not adequately represent the diversity of the original dataset, which can lead to biased performance estimates. The objective of this work is to deepen the investigation of cluster-based cross-validation strategies by analyzing the performance of different clustering algorithms through experimental comparison. Additionally, a new cross-validation technique that combines Mini Batch K-Means with class stratification is proposed. Experiments were conducted on 20 datasets (both balanced and imbalanced) using four supervised learning algorithms, comparing cross-validation strategies in terms of bias, variance, and computational cost. The technique that uses Mini Batch K-Means with class stratification outperformed others in terms of bias and variance on balanced datasets, though it did not significantly reduce computational cost. On imbalanced datasets, traditional stratified cross-validation consistently performed better, showing lower bias, variance, and computational cost, making it a safe choice for performance evaluation in scenarios with class imbalance. In the comparison of different clustering algorithms, no single algorithm consistently stood out as superior. Overall, this work contributes to improving predictive model evaluation strategies by providing a deeper understanding of the potential of cluster-based data splitting techniques and reaffirming the effectiveness of well-established strategies like stratified cross-validation. Moreover, it highlights perspectives for increasing the robustness and reliability of model evaluations, especially in datasets with clustering characteristics.

交叉验证聚类模型评估不平衡数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。