提出后处理方法,让深度回归模型更准地估算不确定度。
Practical Deep Heteroskedastic Regression
- 预训练网络后,用留出数据拟合中间层的方差模型
- 在分子图数据集上不确定性评估达顶尖水平,均值预测不降
- 推理成本极低,适合实际部署
深度学习回归中的不确定性量化(UQ)对序贯决策和风险敏感任务至关重要。在异方差回归中,目标变量的不确定性依赖于输入,通常通过神经网络同时参数化预测分布的均值和方差来实现。然而,训练深度异方差回归模型面临诸多实际挑战,包括优化困难、表征崩溃和方差过拟合。本文揭示了此前未被讨论的误区,并提出一种简单高效的后处理方法:在预训练网络的中间层上,利用留出数据拟合一个方差模型,从而联合解决上述问题。实验表明,该方法在多个分子图数据集上达到相当或超越当前最优的不确定性量化性能,且不损害均值预测精度,推理时开销极小。
原文摘要 · Abstract (English)
Uncertainty quantification (UQ) in deep learning regression is of wide interest, as it supports critical applications including sequential decision making and risk-sensitive tasks. In heteroskedastic regression, where the uncertainty of the target depends on the input, a common approach is to train a neural network that parameterizes the mean and the variance of the predictive distribution. Still, training deep heteroskedastic regression models poses practical challenges in the trade-off between uncertainty quantification and mean prediction, such as optimization difficulties, representation collapse, and variance overfitting. In this work we identify previously undiscussed fallacies and propose a simple and efficient procedure that addresses these challenges jointly by post-hoc fitting a variance model across the intermediate layers of a pretrained network on a hold-out dataset. We demonstrate that our method achieves on-par or state-of-the-art uncertainty quantification on several molecular graph datasets, without compromising mean prediction accuracy and remaining cheap to use at prediction time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。