同时优化预测间隔的均值和方差,提升模型泛化能力。
Hadamard-Riemannian Optimization for Margin-Variance Ensemble
- 将间隔方差纳入损失函数,联合优化期望间隔与方差。
- 在多个基准数据集上优于传统间隔优化方法。
- 通过球面重参数化加速优化,适合大规模场景。
集成学习通过融合多个基模型显著提升预测性能。然而,传统基于间隔的集成方法主要关注最大化期望间隔,忽视了间隔方差的关键作用,这限制了模型的泛化能力,并在噪声或不平衡数据下加剧过拟合风险。此外,传统在概率单纯形上优化集成权重的方法常带来计算低效和可扩展性问题。为此,本文提出一种新框架,将间隔方差显式引入损失函数,联合优化负期望间隔及其方差,从而提升鲁棒性与泛化性能。通过将集成权重重新参数化到单位球面,大幅简化优化过程并提高效率。在多个基准数据集上的大量实验表明,该方法持续优于传统间隔优化技术,验证了其有效性与实用性。
原文摘要 · Abstract (English)
Ensemble learning has been widely recognized as a pivotal technique for boosting predictive performance by combining multiple base models. Nevertheless, conventional margin-based ensemble methods predominantly focus on maximizing the expected margin while neglecting the critical role of margin variance, which inherently restricts the generalization capability of the model and heightens its vulnerability to overfitting, particularly in noisy or imbalanced datasets. Additionally, the conventional approach of optimizing ensemble weights within the probability simplex often introduces computational inefficiency and scalability challenges, complicating its application to large-scale problems. To tackle these limitations, this paper introduces a novel ensemble learning framework that explicitly incorporates margin variance into the loss function. Our method jointly optimizes the negative expected margin and its variance, leading to enhanced robustness and improved generalization performance. Moreover, by reparameterizing the ensemble weights onto the unit sphere, we substantially simplify the optimization process and improve computational efficiency. Extensive experiments conducted on multiple benchmark datasets demonstrate that the proposed approach consistently outperforms traditional margin-based ensemble techniques, underscoring its effectiveness and practical utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。