提出优化分布分割法，提升机器学习评估准确性
Enhancing Automated Machine Learning via Homogeneous Train-Test Splitting Methods
- 将数据分布相似性作为优化目标，改进训练测试划分
- 新方法在15个数据集上平均MMD相似度达89.0%
- 适合处理类别不平衡或空间相关数据的AutoML场景
机器学习模型评估的准确性高度依赖数据集划分方式。传统随机划分假设训练与测试集分布一致，但该假设在存在类别不平衡、自然聚类或空间自相关时常被破坏。本文研究统计相似性在划分中的作用及其对AutoML评估的影响。在15个UCI基准数据集上比较了五种策略：随机划分、分层采样、Kennard-Stone、Duplex和SPXY。使用卡方检验、K-S检验和最大均值差异（MMD）评估相似性。基于几何的方法始终产生接近零的MMD分数，导致下游性能估计不稳定。提出的优化分布（Optimised-Distribution）方法将相似性作为显式优化目标，在所有评估策略中实现最高平均MMD相似度89.0%。
原文摘要 · Abstract (English)
Accurate model evaluation in machine learning depends critically on how datasets are split into training and testing subsets. Standard random splitting assumes that both partitions share the same underlying distribution, an assumption often violated in datasets with class imbalance, natural clustering, or spatial autocorrelation. This paper investigates the role of statistical similarity in train-test splitting and its consequences for AutoML model evaluation. Five established strategies are compared across fifteen UCI benchmark datasets: random splitting, stratified sampling, Kennard-Stone, Duplex, and SPXY. Similarity is assessed using chi-square, Kolmogorov-Smirnov, and Maximum Mean Discrepancy (MMD) tests. Geometry-based methods consistently produce near-zero MMD scores, introducing instability into downstream performance estimates. The proposed Optimised-Distribution method treats similarity as an explicit optimisation objective and achieves the highest mean MMD similarity, 89.0%, across all strategies evaluated.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。