arXiv:2509.25216cs.LGstat.ML2025-09

在基因组预测中发现,双下降现象仅当多维度复杂度协同变化时才出现。

Evaluating Double Descent in Machine Learning: Insights from Tree-Based Models Applied to a Genomic Prediction Task

  • 通过调整树模型容量与集成规模,验证双下降需联合复杂度变化
  • 固定任一维度时,泛化表现回归经典U型或L型曲线
  • 研究支持双下降源于多维复杂度投影的解释,适合机器学习理论研究者

经典学习理论描述了模型复杂度与预测误差之间的典型U型关系,反映从欠拟合到过拟合的转变。近年来,研究提出在过拟合阈值后测试误差再次下降的现象,即双下降。尽管双下降在深度学习中被广泛研究,也已在决策树和梯度提升等简单模型中报告。本文以结核分枝杆菌异烟肼耐药性预测任务为场景,系统地沿两个正交轴(学习器容量,如Pleaf、Pboost;集成规模,即Pens)调节模型复杂度。结果表明,双下降仅在两个轴同时扩展时稳定出现;若任一轴固定,泛化行为则恢复为经典U型或L型。该现象在合成基准上重复验证,支持投影假说——双下降源于不同泛化区域在单一复杂度轴上的投影。研究强调分析泛化行为时应将模型复杂度视为多维概念。所有代码与可复现材料见:https://github.com/guillermocomesanacimadevila/Demystifying-Double-Descent-in-ML。

原文摘要 · Abstract (English)

Classical learning theory describes a well-characterised U-shaped relationship between model complexity and prediction error, reflecting a transition from underfitting in underparameterised regimes to overfitting as complexity grows. Recent work, however, has introduced the notion of a second descent in test error beyond the interpolation threshold-giving rise to the so-called double descent phenomenon. While double descent has been studied extensively in the context of deep learning, it has also been reported in simpler models, including decision trees and gradient boosting. In this work, we revisit these claims through the lens of classical machine learning applied to a biological classification task: predicting isoniazid resistance in Mycobacterium tuberculosis using whole-genome sequencing data. We systematically vary model complexity along two orthogonal axes-learner capacity (e.g., Pleaf, Pboost) and ensemble size (i.e., Pens)-and show that double descent consistently emerges only when complexity is scaled jointly across these axes. When either axis is held fixed, generalisation behaviour reverts to classical U- or L-shaped patterns. These results are replicated on a synthetic benchmark and support the unfolding hypothesis, which attributes double descent to the projection of distinct generalisation regimes onto a single complexity axis. Our findings underscore the importance of treating model complexity as a multidimensional construct when analysing generalisation behaviour. All code and reproducibility materials are available at: https://github.com/guillermocomesanacimadevila/Demystifying-Double-Descent-in-ML.

双下降机器学习理论树模型泛化分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。