arXiv:2602.12469cs.LG2026-02

通过正则化元学习提升集成模型泛化能力,解决冗余、不稳和过拟合问题。

Regularized Meta-Learning for Improved Generalization

  • 四阶段流程:去冗余投影、统计元特征增强、交叉验证正则化建模
  • 在10万样本数据上实现8.582的外折叠RMSE,比简单平均提升3.5%
  • 适合高维集成系统部署,运行速度是贪心搜索的4倍

深度集成方法常能提升预测性能,但存在三大实际问题:基础模型间冗余导致计算成本上升且条件数恶化,多重共线性下权重不稳定,元学习管道易过拟合。本文提出一种正则化元学习框架,通过四阶段流程解决上述挑战:基于相关性与均方误差阈值(τ_corr=0.95)的多指标去重策略,降低元设计矩阵的有效条件数;构建集成统计量与交互项以恢复高阶结构;最后采用逆RMSE加权阶段缓解正则化选择方差。在Playground Series S6E1基准(10万样本,72个基模型)上,该框架取得8.582的外折叠RMSE,优于简单平均(8.894)和传统Ridge堆叠(8.627),接近贪心爬升(8.603)但运行速度快4倍。条件数分析显示去冗余后有效条件数下降53.7%。全面消融实验表明去重、元特征和融合策略均有稳定贡献。结果表明正则化元学习是高维集成系统的稳定且高效堆叠方案。

原文摘要 · Abstract (English)

Deep ensemble methods often improve predictive performance, yet they suffer from three practical limitations: redundancy among base models that inflates computational cost and degrades conditioning, unstable weighting under multicollinearity, and overfitting in meta-learning pipelines. We propose a regularized meta-learning framework that addresses these challenges through a four-stage pipeline combining redundancy-aware projection, statistical meta-feature augmentation, and cross-validated regularized meta-models (Ridge, Lasso, and ElasticNet). Our multi-metric de-duplication strategy removes near-collinear predictors using correlation and MSE thresholds ($τ_{\text{corr}}=0.95$), reducing the effective condition number of the meta-design matrix while preserving predictive diversity. Engineered ensemble statistics and interaction terms recover higher-order structure unavailable to raw prediction columns. A final inverse-RMSE blending stage mitigates regularizer-selection variance. On the Playground Series S6E1 benchmark (100K samples, 72 base models), the proposed framework achieves an out-of-fold RMSE of 8.582, improving over simple averaging (8.894) and conventional Ridge stacking (8.627), while matching greedy hill climbing (8.603) with substantially lower runtime (4 times faster). Conditioning analysis shows a 53.7\% reduction in effective matrix condition number after redundancy projection. Comprehensive ablations demonstrate consistent contributions from de-duplication, statistical meta-features, and meta-ensemble blending. These results position regularized meta-learning as a stable and deployment-efficient stacking strategy for high-dimensional ensemble systems.

集成学习元学习正则化模型融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。