提出蒙特卡洛随机深度法,高效估算深度学习不确定性。
Monte Carlo Stochastic Depth for Uncertainty Estimation in Deep Learning
- 将随机深度正则化改造为贝叶斯近似推断方法
- 在COCO数据集上比MCD略优,校准误差更低
- 适合对不确定性敏感的自动驾驶等安全系统
深度神经网络在安全关键系统中的应用亟需可靠的不确定性量化(UQ)。当前主流策略是将随机正则化方法重用于近似贝叶斯推断,如蒙特卡洛丢弃(MCD)和蒙特卡洛丢块(MCDB)。然而,随机深度(SD)这一现代骨干网络的核心正则化方法尚未被充分探索。尽管先前研究显示其在分割任务中具潜力,但缺乏与变分贝叶斯推断的理论联系,且未在复杂多任务场景(如目标检测)中进行基准测试。本文首次建立蒙特卡洛随机深度(MCSD)与严格变分推断的理论关联,并在YOLO、RT-DETR等先进检测器上,基于COCO和COCO-O数据集进行全面实证评估。结果表明,MCSD在保持高预测精度(mAP)的同时,显著提升校准性能(ECE)和不确定性排序能力(AUARC),优于MCD。因此,MCSD成为现代深度学习中兼具理论基础与实证有效性的高效贝叶斯近似工具。
原文摘要 · Abstract (English)
The deployment of deep neural networks in safety-critical systems necessitates reliable and efficient uncertainty quantification (UQ). A practical and widespread strategy for UQ is repurposing stochastic regularizers as scalable approximate Bayesian inference methods, such as Monte Carlo Dropout (MCD) and MC-DropBlock (MCDB). However, this paradigm remains under-explored for Stochastic Depth (SD), a regularizer integral to the residual-based backbones of most modern architectures. While prior work demonstrated its empirical promise for segmentation, a formal theoretical connection to Bayesian variational inference and a benchmark on complex, multi-task problems like object detection are missing. In this paper, we first provide theoretical insights connecting Monte Carlo Stochastic Depth (MCSD) to principled approximate variational inference. We then present the first comprehensive empirical benchmark of MCSD against MCD and MCDB on state-of-the-art detectors (YOLO, RT-DETR) using the COCO and COCO-O datasets. Our results position MCSD as a robust and computationally efficient method that achieves highly competitive predictive accuracy (mAP), notably yielding slight improvements in calibration (ECE) and uncertainty ranking (AUARC) compared to MCD. We thus establish MCSD as a theoretically-grounded and empirically-validated tool for efficient Bayesian approximation in modern deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。