通过实时估计分布偏移概率,更准确评估视觉深度学习系统的可靠性。
Probabilistic Runtime Verification, Evaluation and Risk Assessment of Visual Deep Learning Systems
- 用分布外检测器输出估算输入偏移概率,构建二叉树结构建模正确性
- 在五个数据集上测试,误差范围0.01至0.1,优于传统评估方法
- 可应用于医疗分割等高风险场景,支持成本收益分析与决策判断
尽管在基准测试中表现优异,深度神经网络在真实部署中常因对输入数据微小但通常不可察觉的分布偏移敏感而性能下降。这类偏移在实际场景中普遍存在,但评估时极少被考虑,导致性能指标虚高。为此,我们提出一种新的深度学习系统验证、评估与风险评估方法。该方法通过分布外检测器输出,实时估算分布偏移发生的概率,并将其与网络正确性的条件概率结合,构建二叉树结构。通过遍历此树,可计算出可信且精确的网络准确率估计。我们在五个不同数据集上模拟了具有不同分布偏移频率的部署条件进行评估,结果表明该方法始终优于传统评估,准确率估计误差通常在0.01至0.1之间。此外,我们在医学分割基准上展示了该方法在风险评估中的潜力,通过为树节点赋值成本,支持成本-收益分析与价值判断。最终,该方法为提升深度学习系统在安全关键应用中的可靠性与可信度提供了稳健框架,实现更精准的性能估计与可操作的风险评估。
原文摘要 · Abstract (English)
Despite achieving excellent performance on benchmarks, deep neural networks often underperform in real-world deployment due to sensitivity to minor, often imperceptible shifts in input data, known as distributional shifts. These shifts are common in practical scenarios but are rarely accounted for during evaluation, leading to inflated performance metrics. To address this gap, we propose a novel methodology for the verification, evaluation, and risk assessment of deep learning systems. Our approach explicitly models the incidence of distributional shifts at runtime by estimating their probability from outputs of out-of-distribution detectors. We combine these estimates with conditional probabilities of network correctness, structuring them in a binary tree. By traversing this tree, we can compute credible and precise estimates of network accuracy. We assess our approach on five different datasets, with which we simulate deployment conditions characterized by differing frequencies of distributional shift. Our approach consistently outperforms conventional evaluation, with accuracy estimation errors typically ranging between 0.01 and 0.1. We further showcase the potential of our approach on a medical segmentation benchmark, wherein we apply our methods towards risk assessment by associating costs with tree nodes, informing cost-benefit analyses and value-judgments. Ultimately, our approach offers a robust framework for improving the reliability and trustworthiness of deep learning systems, particularly in safety-critical applications, by providing more accurate performance estimates and actionable risk assessments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。