arXiv:2411.17767cs.CVcs.LG2024-11被引 6

用视觉基础模型量化目标检测中的数据不确定性,提升训练可靠性。

Exploring Aleatoric Uncertainty in Object Detection via Vision Foundation Models

  • 基于视觉基础模型的特征空间,用马氏距离估计每类物体的随机不确定性。
  • 在多个检测模型和挑战性数据集上,显著提升检测性能与鲁棒性。
  • 无需修改模型结构,可作为即插即用的额外标注,适合数据质量差的场景。

开放世界数据不可避免地包含各种随机噪声,导致目标检测中普遍存在数据不确定性(aleatoric uncertainty)。与分类任务中中心化、相似尺度的图像不同,目标检测面临多尺度、遮挡、模糊及标注噪声等问题。本文提出利用视觉基础模型建模并利用数据中的内在不确定性,构建以数据为中心的可靠训练范式。技术上,基于在超大规模数据集上训练的视觉基础模型特征空间,假设物体特征服从混合高斯分布,采用马氏距离度量数据不确定性。进一步提出两种实用方法:1)构建不确定性感知样本筛选器,剔除噪声和冗余样本,防止过拟合;2)设计样本自适应正则化项,平衡难易样本,实现自适应训练。所估计的不确定性可作为数据集的额外标注,兼容任意检测模型。大量实验证明该方法在多种先进检测模型和挑战性基准上均有效。

原文摘要 · Abstract (English)

Datasets collected from the open world unavoidably suffer from various forms of randomness or noiseness, leading to the ubiquity of aleatoric (data) uncertainty. Quantifying such uncertainty is particularly pivotal for object detection, where images contain multi-scale objects with occlusion, obscureness, and even noisy annotations, in contrast to images with centric and similar-scale objects in classification. This paper suggests modeling and exploiting the uncertainty inherent in object detection data with vision foundation models and develops a data-centric reliable training paradigm. Technically, we propose to estimate the data uncertainty of each object instance based on the feature space of vision foundation models, which are trained on ultra-large-scale datasets and able to exhibit universal data representation. In particular, we assume a mixture-of-Gaussian structure of the object features and devise Mahalanobis distance-based measures to quantify the data uncertainty. Furthermore, we suggest two curial and practical usages of the estimated uncertainty: 1) for defining uncertainty-aware sample filter to abandon noisy and redundant instances to avoid over-fitting, and 2) for defining sample adaptive regularizer to balance easy/hard samples for adaptive training. The estimated aleatoric uncertainty serves as an extra level of annotations of the dataset, so it can be utilized in a plug-and-play manner with any model. Extensive empirical studies verify the effectiveness of the proposed aleatoric uncertainty measure on various advanced detection models and challenging benchmarks.

目标检测不确定性视觉基础模型数据清洗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。