arXiv:2511.17760cond-mat.mtrl-scics.LG2025-11被引 3

主动学习中不确定性估计失效,可能因分布外预测不准确。

When Active Learning Fails, Uncalibrated Out of Distribution Uncertainty Quantification Might Be the Problem

  • 用多种模型对比了不同不确定性估计方法的效果。
  • 校准后的不确定性未能减少模型在分布外数据上的训练数据需求。
  • 问题根源或在于输入特征空间中的经验不确定性建模不足。

在材料发现的主动学习中,高效准确地估计预测不确定性对探索至关重要,高不确定性样本被视为模型缺失信息的来源。本文评估了基于ALIGNN、XGBoost、随机森林和神经网络集成模型的多种不确定性估计与校准方法在主动学习中的表现,比较了深度集成与损失景观不确定性估计在溶解度、带隙和生成能预测任务中的效果。结果表明,不确定性校准方法在从域内数据泛化到域外数据时表现不一;且相较于随机采样和未校准不确定性,校准后的不确定性并未有效降低模型在域外数据上改进所需的样本量。该问题在随机森林和梯度提升模型中同样存在,表明其部分源于数据特性而非模型容量。对目标分布、域内/域外不确定性及训练残差分布的分析提示,未来工作应关注输入特征空间中的经验不确定性建模,以解决集成预测方差无法准确捕捉模型泛化所需信息的问题。

原文摘要 · Abstract (English)

Efficiently and meaningfully estimating prediction uncertainty is important for exploration in active learning campaigns in materials discovery, where samples with high uncertainty are interpreted as containing information missing from the model. In this work, the effect of different uncertainty estimation and calibration methods are evaluated for active learning when using ensembles of ALIGNN, eXtreme Gradient Boost, Random Forest, and Neural Network model architectures. We compare uncertainty estimates from ALIGNN deep ensembles to loss landscape uncertainty estimates obtained for solubility, bandgap, and formation energy prediction tasks. We then evaluate how the quality of the uncertainty estimate impacts an active learning campaign that seeks model generalization to out-of-distribution data. Uncertainty calibration methods were found to variably generalize from in-domain data to out-of-domain data. Furthermore, calibrated uncertainties were generally unsuccessful in reducing the amount of data required by a model to improve during an active learning campaign on out-of-distribution data when compared to random sampling and uncalibrated uncertainties. The impact of poor-quality uncertainty persists for random forest and eXtreme Gradient Boosting models trained on the same data for the same tasks, indicating that this is at least partially intrinsic to the data and not due to model capacity alone. Analysis of the target, in-distribution uncertainty, out-of-distribution uncertainty, and training residual distributions suggest that future work focus on understanding empirical uncertainties in the feature input space for cases where ensemble prediction variances do not accurately capture the missing information required for the model to generalize.

主动学习不确定性分布外

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。