提出可调误差分解框架,精细控制模型性能与多样性平衡。
Self-Error Adjustment: Theory and Practice of Balancing Individual Performance and Diversity in Ensemble Learning
- 将集成误差拆分为个体误差与交互多样性两部分,引入可调参数控制
- 在多个数据集上优于传统方法,性能提升稳定且可调范围更广
- 理论证明更紧的边界,适合需要精细调参的集成学习场景
集成学习通过聚合多个基础模型的预测结果来提升性能,核心挑战在于平衡个体模型精度与多样性。传统方法如Bagging和Boosting通过随机性促进多样性,但对精度-多样性权衡缺乏精确控制。负相关学习(NCL)虽引入惩罚项调节该权衡,但理论边界松散、调节范围有限。为此,本文提出自误差调节(Self-Error Adjustment, SEA)新框架,将集成误差分解为个体性能项(反映各模型自误差)与多样性项(反映模型间交互)。该分解使损失函数中可引入可调参数,实现对两类分量贡献的精确调控,从而更精细地调节集成性能。相较于NCL及其变体,SEA具备更广的有效调节范围与更一致的多样性变化。同时,本文建立了更紧的可调集成方法理论边界,并通过实验证实其有效性。在多个公开回归与分类数据集上的实验表明,SEA在所有任务中均持续优于基线方法。消融研究进一步验证了其更强的灵活性与调参优势。
原文摘要 · Abstract (English)
Ensemble learning boosts performance by aggregating predictions from multiple base learners. A core challenge is balancing individual learner accuracy with diversity. Traditional methods like Bagging and Boosting promote diversity through randomness but lack precise control over the accuracy-diversity trade-off. Negative Correlation Learning (NCL) introduces a penalty to manage this trade-off but suffers from loose theoretical bounds and limited adjustment range. To overcome these limitations, we propose a novel framework called Self-Error Adjustment (SEA), which decomposes ensemble errors into two distinct components: individual performance terms, representing the self-error of each base learner, and diversity terms, reflecting interactions among learners. This decomposition allows us to introduce an adjustable parameter into the loss function, offering precise control over the contribution of each component, thus enabling finer regulation of ensemble performance. Compared to NCL and its variants, SEA provides a broader range of effective adjustments and more consistent changes in diversity. Furthermore, we establish tighter theoretical bounds for adjustable ensemble methods and validate them through empirical experiments. Experimental results on several public regression and classification datasets demonstrate that SEA consistently outperforms baseline methods across all tasks. Ablation studies confirm that SEA offers more flexible adjustment capabilities and superior performance in fine-tuning strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。