提出新损失函数,让模型更好捕捉预测不确定性的双峰分布。
Beyond the Mean: Distribution-Aware Loss Functions for Bimodal Regression
- 用归一化RMSE结合Wasserstein和Cramér距离设计新损失函数。
- 在复杂双峰数据上,JS散度降低45%,优于标准回归和MDN。
- 适合需要可信不确定性估计的AI系统,如医疗、自动驾驶。
尽管机器学习模型在诸多领域表现出色,但如何通过可靠的预测置信度评估其可信性仍是一大挑战。当预测误差分布呈现双峰特征时——源于自信与模糊预测并存——传统回归方法因假设单峰高斯噪声,常出现均值坍缩。虽然混合密度网络(MDNs)可建模多分布,但优化极不稳定。本文提出一族分布感知损失函数,融合归一化均方根误差与Wasserstein及Cramér距离。应用于标准深度回归模型后,无需混合模型即可恢复双峰分布。四阶段实验验证表明,所提Wasserstein损失构建了新的帕累托效率前沿:在单峰任务中保持与MSE相当的稳定性,同时在复杂双峰数据集上使Jensen-Shannon散度降低45%。该框架在保真度与鲁棒性上全面超越MDNs,为可信AI系统中的偶然不确定性估计提供可靠工具。
原文摘要 · Abstract (English)
Despite the strong predictive performance achieved by machine learning models across many application domains, assessing their trustworthiness through reliable estimates of predictive confidence remains a critical challenge. This issue arises in scenarios where the likelihood of error inferred from learned representations follows a bimodal distribution, resulting from the coexistence of confident and ambiguous predictions. Standard regression approaches often struggle to adequately express this predictive uncertainty, as they implicitly assume unimodal Gaussian noise, leading to mean-collapse behavior in such settings. Although Mixture Density Networks (MDNs) can represent different distributions, they suffer from severe optimization instability. We propose a family of distribution-aware loss functions integrating normalized RMSE with Wasserstein and Cramér distances. When applied to standard deep regression models, our approach recovers bimodal distributions without the volatility of mixture models. Validated across four experimental stages, our results show that the proposed Wasserstein loss establishes a new Pareto efficiency frontier: matching the stability of standard regression losses like MSE in unimodal tasks while reducing Jensen-Shannon Divergence by 45% on complex bimodal datasets. Our framework strictly dominates MDNs in both fidelity and robustness, offering a reliable tool for aleatoric uncertainty estimation in trustworthy AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。