用解析方法提升深度集成模型的效率与可解释性
Efficient Bayesian Deep Ensembles via Analytic Predictive Inference

- 用少量神经网络组合预测,推理成本仅随集成规模增长
- 通过闭式贝叶斯聚合,直接得到可解释的权重和校准的不确定性
- 独立训练多个网络,增强鲁棒性,适合需要可靠置信度的场景
我们提出一种高效的贝叶斯深度集成方法,用于回归任务,旨在提升可解释性的同时保持优异的预测性能与计算效率。该方法结合贝叶斯推断的统计严谨性与深度集成的可扩展性,提供校准的不确定性估计,不仅可用于独立预测,还可作为更复杂学习系统的一部分。核心设计包括:(i) 低维集成表示:预测由少量训练好的神经网络组合而成,实现可扩展推理,成本仅依赖于集成规模而非数据集大小;(ii) 闭式贝叶斯聚合:使用贝叶斯线性回归合并集成预测,获得可解释的后验权重和校准的不确定性,无需近似推断;(iii) 独立集成训练:多个神经网络分别训练,生成多样化的预测表示,提升鲁棒性和不确定性校准能力。在标准回归基准上的实验证明,该方法在保持可靠不确定性估计的同时,实现了具有竞争力的预测性能。
原文摘要 · Abstract (English)
We introduce an efficient Bayesian deep ensemble method for predictive regression designed to enhance interpretability while maintaining competitive predictive performance and computational efficiency. Our method combines the statistical rigor of Bayesian inference with the scalability of deep ensembles, providing calibrated uncertainty estimates that enable its use not only for standalone prediction but also as a component within broader learning systems. To achieve these goals, our work relies on three key design components: (i) low-dimensional ensemble representation: predictions are expressed as a combination of a small number of trained neural predictors, enabling scalable inference whose cost depends on ensemble size rather than dataset size; (ii) closed-form Bayesian aggregation: ensemble predictions are combined using Bayesian linear regression, yielding interpretable posterior weights and calibrated uncertainty without approximate inference; and (iii) Independent ensemble training: multiple neural networks are trained separately, producing diverse predictive representations that improve robustness and uncertainty calibration. Empirical results on standard regression benchmarks demonstrate that the proposed approach achieves competitive predictive performance while maintaining reliable uncertainty estimates across settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。