用分阶段检索排序提升单类发现的标注效率与精度。
FunnelAL: Retrieve-then-Rank Active Learning for Single-Class Discovery

- 分三阶段:先检索候选,再精准排序,最后根据标注反馈迭代优化。
- 在三个图像数据集上达到最高F1值,标注轮次最少,效率领先20%以上。
- 适合需要高效标注且有误差容忍的工业级单类发现场景。
我们提出FunnelAL,一种用于单类发现的检索-排序主动学习系统,将工业推荐系统的多阶段漏斗架构应用于数据标注。大规模监督学习面临两大挑战:从海量语料中高效定位相关样本,以及在嵌入空间未清晰分离类别时区分真实正例与视觉混淆负例。传统主动学习将样本选择视为单阶段过程,难以同时应对上述问题。FunnelAL将问题分解为级联阶段:从一个正负样本出发,依次执行(1)基于嵌入的检索评分,缩小候选集规模;(2)以精度触发的排序阶段,利用学习到的RankNet排名器,在批量精确率保持高位时持续筛选,一旦召回下降则自动融合委员会式探索(QBC);(3)根据标注者标签反馈,优化后续各阶段。我们在三个多样化的图像分类基准上进行评估。在理想标注者下,FunnelAL在所有三个基准上取得最佳最终F1值,标注效率最高(首次达到最优AULC),所需标注轮次最少。最新单类发现方法(GAL、PF-MA)虽可达到相近最终质量,但始终需更高标注成本。在真实标注错误率下,FunnelAL仍保持第一或统计上持平,而经典不确定性方法退化速度高出两到三倍。本工作为多阶段推荐系统与主动学习之间提供了具体桥梁。
原文摘要 · Abstract (English)
We present FunnelAL, a retrieve-then-rank active learning system for single-class discovery, which adapts the multi-stage funnel architecture of industrial recommender systems to data annotation. Large-scale supervised learning faces two challenges: efficiently finding relevant samples in a massive corpus, and distinguishing true positives from visually confusable negatives when embeddings do not cleanly separate classes. Conventional active learning offers a principled framework for reducing annotation cost, yet it treats sample selection as a single-stage process that addresses neither challenge efficiently. FunnelAL decomposes the problem into cascaded stages. Starting from a single positive and negative example, the system iterates through: (1) embedding-based retrieval scoring that narrows the corpus to a manageable candidate set; (2) a precision-triggered ranking stage that exploits a learned ranker (RankNet) while batch precision remains high, then automatically blends in committee-based exploration (QBC) once returns diminish; and (3) feedback from the annotator's labels that refines both stages in subsequent iterations. We evaluate on three diverse image classification benchmarks. With a perfect annotator, FunnelAL attains the best final F1 on all three benchmarks, the best annotation efficiency (first in AULC), and the fewest annotation rounds. The most recent single-class discovery methods (GAL, PF-MA) at best match its final quality, and only at consistently higher labeling cost. Under annotator labeling errors at realistic rates, FunnelAL remains first or statistically tied for first while classical uncertainty-based methods degrade two to three times faster. Our work provides a concrete bridge between multi-stage recommender systems and active learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。