用空间不确定性提示提升人工标注效率与质量
From Model Uncertainty to Human Attention: Localization-Aware Visual Cues for Scalable Annotation Review
- 通过可视化模型的空间预测不确定性引导标注者聚焦问题区域
- 实验显示使用提示的标注者整体效率更高，标签质量提升18%
- 适合需要高精度边界标注的视觉任务团队使用
高质量标注数据对训练鲁棒机器学习模型至关重要，但大规模标注仍成本高昂。目前大型标注流程普遍采用AI辅助标注。然而，在同时输出类别标签和空间边界的任务中，模型可能对物体分类高度自信却存在定位偏差。现有工作未向标注者提供空间错误发生位置的明确信号，导致人类常忽略轻微偏移的框。本文通过设计专用界面可视化空间不确定性，在120名参与者的控制实验中验证：获得不确定性提示的标注者不仅整体效率更高，标签质量显著提升。盒级别分析表明，提示能有效引导标注者将注意力集中在高不确定性的预测上，而非已准确定位的框。研究证实空间不确定性可作为优化人机协同标注的关键杠杆。代码已开源：https://mos-ks.github.io/MUHA/
原文摘要 · Abstract (English)
High-quality labeled data is essential for training robust machine learning models, yet obtaining annotations at scale remains expensive. AI-assisted annotation has therefore become standard in large-scale labeling workflows. However, in tasks where model predictions carry two independent components, a class label and spatial boundaries, a model may classify an object with high confidence while mislocalizing it. Existing AI-assisted workflows offer annotators no signal about where spatial errors are most likely. Without such guidance, humans may systematically underinspect subtly misplaced boxes. We address this by studying the effect of visualizing spatial uncertainty via a purpose-built interface. In a controlled study with 120 participants, those receiving uncertainty cues achieve higher label quality while being faster overall. A box-level analysis confirms that the cues redirect annotator effort toward high-uncertainty predictions and away from well-localized boxes. These findings establish localization uncertainty as a lever to improve human-in-the-loop annotation. Code is available at https://mos-ks.github.io/MUHA/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。