arXiv:2503.05538cs.LGstat.ML2025-03

揭示提升型可加模型的解路径与理论局限,为实际应用提供关键指导

Additive Model Boosting: New Insights and Path(ologie)s

  • 通过分析解路径,建立提升型可加模型与其它方法的联系
  • 提出新收敛性结果,证明方法在多数情况有效但存在特定病态行为
  • 适合关注模型可解释性与优化机制的研究者参考

可加模型(AMs)近年来在机器学习中备受关注,能将可解释结构融入多种模型类别。许多常用方法基于提升可加模型(BAMs)思想,适用于复杂可加模型拟合。尽管BAMs在实践中表现良好,但其理论特性仍不清晰,包括整体收敛行为及提升隐含正则化下实际求解的优化问题。本文研究BAMs的解路径,揭示其与特定问题类别的其他方法间的关联。我们推导出BAMs的新收敛结果,深入揭示方法内在机制。研究结果总体上为BAMs的实际使用提供了令人安心的理论支持,但也发现某些可加模型类别中提升法存在收敛异常,需在实践中谨慎对待。通过多个数值实验验证了理论发现。

原文摘要 · Abstract (English)

Additive models (AMs) have sparked a lot of interest in machine learning recently, allowing the incorporation of interpretable structures into a wide range of model classes. Many commonly used approaches to fit a wide variety of potentially complex additive models build on the idea of boosting additive models. While boosted additive models (BAMs) work well in practice, certain theoretical aspects are still poorly understood, including general convergence behavior and what optimization problem is being solved when accounting for the implicit regularizing nature of boosting. In this work, we study the solution paths of BAMs and establish connections with other approaches for certain classes of problems. Along these lines, we derive novel convergence results for BAMs, which yield crucial insights into the inner workings of the method. While our results generally provide reassuring theoretical evidence for the practical use of BAMs, they also uncover some ``pathologies'' of boosting for certain additive model classes concerning their convergence behavior that require caution in practice. We empirically validate our theoretical findings through several numerical experiments.

可加模型提升法收敛性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。