用置信框架增强集成学习,让预测区间更可靠且无需大量计算。
Conformalized Super Learner
- 基于原始集成框架,用加权投票融合各模型的符合度得分构建区间。
- 在有限样本下保证覆盖概率,对异方差、稀疏等复杂数据表现良好。
- 适合需要可信预测区间的医疗等高风险领域,如肾功能预测。
Super Learner(SL)是一种广泛使用的集成方法,通过结合多个学习器的预测结果来提升性能。区间预测对量化个体或集成模型的不确定性具有重要意义。已有方法多依赖渐近理论或计算量大的自助法。本文提出将置信预测(Conformal Prediction, CP)与SL结合,采用自然构造:利用各学习器权重,并通过加权多数投票方式整合学习器特定的符合度得分。我们分析了该方法在连续结果下的性质,涵盖交换性假设、其破坏情形以及存在异方差、稀疏性等分布异质性的数据生成机制。综合模拟研究表明,该方法在有限样本下实现有效覆盖,性能接近真实数据生成机制。核心贡献是应用于使用社会人口、生物特征和实验室测量预测肌酐水平的实例,展示了精心选择的学习器组合在捕捉非线性效应、交互作用、稀疏性、异方差及抗异常值方面的优势。
原文摘要 · Abstract (English)
The Super Learner (SL) is a widely used ensemble method that combines predictions from a library of learners based on their predictive performance. Interval predictions are of considerable practical interest because they allow uncertainty in predictions produced by an individual learner or an ensemble to be quantified. Several methods have been proposed for constructing interval predictions based on the SL, however, these approaches are typically justified using asymptotic arguments or rely on computationally intensive procedures such as the bootstrap. Conformal prediction (CP) is a machine learning framework for constructing prediction intervals with finite-sample and asymptotic coverage guarantees under mild conditions. We propose coupling CP with the SL through a natural construction that mirrors the original SL framework, using individual learner weights and combining learner-specific conformity scores via a weighted majority vote. We characterize the properties of the resulting SL-based prediction intervals for continuous outcomes. We cover settings under exchangeability, potential violations of exchangeability, and data-generating mechanisms exhibiting heteroscedasticity, sparsity, and other forms of distributional heterogeneity. A comprehensive simulation study shows that the conformalized SL achieves valid finite-sample coverage with competitive performance relative to the true data-generating mechanism. A central contribution of this work is an application to predicting creatinine levels using socio-demographic, biometric, and laboratory measurements. This example demonstrates the benefits of an ensemble with carefully selected learners designed to capture key aspects of complex regression functions, including non-linear effects, interactions, sparsity, heteroscedasticity, and robustness to outliers.R
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。