动态调用医生标注，减少一半人力却保持高准度。
Optimizing Resources for On-the-Fly Label Estimation with Multiple Unknown Medical Experts
- 根据任务难易实时决定是否追加专家意见。
- 比固定采样少50%标注请求，准确率相当。
- 适合数据连续到来、医生水平未知的医疗筛查场景。
医疗筛查中准确估计真实标签常依赖多位专家协作与同行复核。高效聚合噪声标注的算法可提升筛查流程效率，尤其在数据持续流入且专家能力未知时。现有方法难以无缝集成到筛查流水线。为此，我们提出一种自适应实时标注方法：（一）支持对流式数据即时标注；（二）无需预先知晓专家信息或已有标注数据；（三）根据实例隐含难度动态调用额外专家。该方法逐次收集专家意见直至达到置信阈值，以较低标注开销实现精准标签生成。我们在三个跨模态多标注分类数据集上评估，结果表明，所提自适应查询策略最多减少50%专家调用次数，同时保持与非自适应基线相当的准确率。代码已开源：https://github.com/tbary/MEDICS。
原文摘要 · Abstract (English)
Accurate ground truth estimation in medical screening programs often relies on coalitions of experts and peer second opinions. Algorithms that efficiently aggregate noisy annotations can enhance screening workflows, particularly when data arrive continuously and expert proficiency is initially unknown. However, existing algorithms do not meet the requirements for seamless integration into screening pipelines. We therefore propose an adaptive approach for real-time annotation that (I) supports on-the-fly labeling of incoming data, (II) operates without prior knowledge of medical experts or pre-labeled data, and (III) dynamically queries additional experts based on the latent difficulty of each instance. The method incrementally gathers expert opinions until a confidence threshold is met, providing accurate labels with reduced annotation overhead. We evaluate our approach on three multi-annotator classification datasets across different modalities. Results show that our adaptive querying strategy reduces the number of expert queries by up to 50% while achieving accuracy comparable to a non-adaptive baseline. Our code is available at https://github.com/tbary/MEDICS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。