让集成模型自动淘汰弱分类器，提升准确率与可信度。
Simplex-Constrained Sparse Bagging: Transitioning from Uniform Priors to Sparse Posteriors in Ensemble Learning
- 在单纯形约束下优化投票权重，实现模型压缩与校准
- 最高可压缩96%模型体积，同时降低校准误差
- 适用于任意基学习器，适合部署在资源受限场景
我们提出简约型稀疏袋装法（SCSB），一种数学严谨的后训练压缩与概率校准框架，用于基于自助采样的集成学习。标准袋装集成（如随机森林、袋装SVM、袋装神经网络）对所有基学习器赋予均等投票权，但这种简单先验忽略了其局部性能差异，导致模型过度自信。我们将集成剪枝与校准建模为在概率单纯形上的联合优化问题，通过最小化袋外（OOB）损失实现。为引入稀疏性，我们解决了理论上的“L1单纯形悖论”——即在单纯形上L1范数恒定无法实现剪枝——通过引入凹二次惩罚项。SCSB具有模型无关性，最高可实现96%的集成压缩，带来线性推理加速，并显著改善概率校准（降低期望校准误差），同时保持或提升泛化性能。
原文摘要 · Abstract (English)
We present Simplex-Constrained Sparse Bagging (SCSB), a mathematically rigorous framework for post-training compression and probability calibration of bootstrap-based bagging ensembles. Standard bagging ensembles (such as Random Forests, Bagged SVMs, and Bagged Neural Networks) assign uniform voting power to all constituent estimators. However, this naive uniform prior ignores the varying local competence of base estimators and contributes to model overconfidence. We formulate ensemble pruning and calibration as a joint optimization problem over the probability simplex by minimizing the Out-Of-Bag (OOB) loss. To induce sparsity, we address the theoretical "L1-simplex paradox" - the mathematical reality that the L1 norm is constant on the simplex and fails to prune - by introducing a concave quadratic penalty. SCSB is model-agnostic and achieves up to 96% ensemble compression, yielding linear inference speedups and superior probability calibration (lowered Expected Calibration Error) while preserving or enhancing generalization accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。