arXiv:2412.07169cs.LGcs.CV2024-12CVPR被引 12

动态调整推理时丢弃率,让神经网络更准地判断自身不确定性。

Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation

  • 根据每层特征图的信息损失,实时调节丢弃率。
  • 在医学影像任务中显著提升不确定性估计的校准度。
  • 无需标签,适合医疗等高风险场景的模型可靠性增强。

准确的不确定性估计对医疗诊断等高风险应用至关重要。蒙特卡洛丢弃法通过推理时随机前向传播来近似预测不确定性,但固定各层和输入的丢弃率会导致次优结果,因无法适应不同输入和层的特性。现有方法在训练中优化丢弃率,但导致推理时参数固定,难以应对新数据分布,影响蒙特卡洛模拟的不确定性估计。本文提出 Rate-In,一种推理时动态调整丢弃率的方法,通过量化每层特征图因丢弃产生的信息损失,将丢弃视为可控噪声注入,基于信息论原则实现按层、按输入实例的自适应调整,无需真实标签。通过衡量特征图的功能信息损失,动态调优丢弃率以保持多样医学影像任务中的感知质量。大量实验证明,与固定或启发式丢弃率相比,Rate-In 在不牺牲预测性能的前提下,显著改善了校准效果并锐化了不确定性估计。该方法为关键应用提供了实用、无监督、推理时优化的丢弃策略。

原文摘要 · Abstract (English)

Accurate uncertainty estimation is crucial for deploying neural networks in risk-sensitive applications such as medical diagnosis. Monte Carlo Dropout is a widely used technique for approximating predictive uncertainty by performing stochastic forward passes with dropout during inference. However, using static dropout rates across all layers and inputs can lead to suboptimal uncertainty estimates, as it fails to adapt to the varying characteristics of individual inputs and network layers. Existing approaches optimize dropout rates during training using labeled data, resulting in fixed inference-time parameters that cannot adjust to new data distributions, compromising uncertainty estimates in Monte Carlo simulations. In this paper, we propose Rate-In, an algorithm that dynamically adjusts dropout rates during inference by quantifying the information loss induced by dropout in each layer's feature maps. By treating dropout as controlled noise injection and leveraging information-theoretic principles, Rate-In adapts dropout rates per layer and per input instance without requiring ground truth labels. By quantifying the functional information loss in feature maps, we adaptively tune dropout rates to maintain perceptual quality across diverse medical imaging tasks and architectural configurations. Our extensive empirical study on synthetic data and real-world medical imaging tasks demonstrates that Rate-In improves calibration and sharpens uncertainty estimates compared to fixed or heuristic dropout rates without compromising predictive performance. Rate-In offers a practical, unsupervised, inference-time approach to optimizing dropout for more reliable predictive uncertainty estimation in critical applications.

不确定性估计医学影像自适应丢弃信息论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。