提出量化主动学习效率的新指标，能准确衡量少样本达到同等效果的能力。
The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metric
- 引入速度提升因子，量化主动学习中达成目标性能所需样本比例。
- 在四个数据集上验证，该指标能稳定反映不同迭代阶段的样本效率。
- 适合关注主动学习评估方法的研究者与工业界模型优化团队。
机器学习模型依赖大量标注数据表现优异，但标注成本高且耗时。主动学习（AL）通过查询策略（QMs）迭代选择最具信息量的样本，以提升性能与标注成本的比率。尽管研究集中于新查询策略开发，但对迭代过程的评估缺乏合适指标。本文回顾八年主动学习评估文献，正式提出速度提升因子，作为量化多轮主动学习中查询策略性能的指标，反映达成随机采样性能所需的样本比例。基于四个跨领域数据集和七种不同类型查询策略，实验验证了该指标的有效性：结果证实其假设成立，准确捕捉预期样本比例，并在多轮迭代中表现出更优稳定性。
原文摘要 · Abstract (English)
Machine learning models excel with abundant annotated data, but annotation is often costly and time-intensive. Active learning (AL) aims to improve the performance-to-annotation ratio by using query methods (QMs) to iteratively select the most informative samples. While AL research focuses mainly on QM development, the evaluation of this iterative process lacks appropriate performance metrics. This work reviews eight years of AL evaluation literature and formally introduces the speed-up factor, a quantitative multi-iteration QM performance metric that indicates the fraction of samples needed to match random sampling performance. Using four datasets from diverse domains and seven QMs of various types, we empirically evaluate the speed-up factor and compare it with state-of-the-art AL performance metrics. The results confirm the assumptions underlying the speed-up factor, demonstrate its accuracy in capturing the described fraction, and reveal its superior stability across iterations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。