教人正确评估机器学习模型,避免常见陷阱
Evaluating Supervised Machine Learning Models: Principles, Pitfalls, and Metric Selection

- 强调评估应结合任务目标和数据特点,而非只看单一指标
- 指出准确率陷阱、数据泄露等问题会误导模型表现判断
- 适合想提升模型评估严谨性的研究者和工程师
监督学习模型的评估是构建可靠预测系统的关键阶段。尽管机器学习工具库普及,模型评估仍常简化为少数汇总指标,可能误导对真实性能的判断。本文探讨分类与回归任务中评估的原则、挑战与实践考量,分析数据特征、验证设计、类别不平衡、误差代价不对称及指标选择如何影响评估结果。通过多组基准数据集的受控实验,揭示了准确率悖论、数据泄露、指标误选和过度依赖单一数值等常见陷阱。文章还对比不同验证策略,强调评估需与任务实际应用目标对齐。通过将评估视为决策导向且情境依赖的过程,本文为选择科学、稳健、可信的评估方法提供结构化指导。
原文摘要 · Abstract (English)
The evaluation of supervised machine learning models is a critical stage in the development of reliable predictive systems. Despite the widespread availability of machine learning libraries and automated workflows, model assessment is often reduced to the reporting of a small set of aggregate metrics, which can lead to misleading conclusions about real-world performance. This paper examines the principles, challenges, and practical considerations involved in evaluating supervised learning algorithms across classification and regression tasks. In particular, it discusses how evaluation outcomes are influenced by dataset characteristics, validation design, class imbalance, asymmetric error costs, and the choice of performance metrics. Through a series of controlled experimental scenarios using diverse benchmark datasets, the study highlights common pitfalls such as the accuracy paradox, data leakage, inappropriate metric selection, and overreliance on scalar summary measures. The paper also compares alternative validation strategies and emphasizes the importance of aligning model evaluation with the intended operational objective of the task. By presenting evaluation as a decision-oriented and context-dependent process, this work provides a structured foundation for selecting metrics and validation protocols that support statistically sound, robust, and trustworthy supervised machine learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。