arXiv:2507.15381cs.LGcs.AI2025-07ICCV被引 2

提出PALM模型,可预测主动学习在少量标注下的表现。

To Label or Not to Label: PALM -- A Predictive Model for Evaluating Sample Efficiency in Active Learning Models

  • 构建数学模型,用四个参数描述主动学习动态过程
  • 仅需少量标注数据即可准确预测完整学习曲线
  • 帮助选最优策略,尤其适合预算紧张的场景

主动学习旨在通过选择最具信息量的样本减少标注成本,尤其适用于资源受限场景。然而,传统评估方法仅关注最终准确率,无法捕捉学习过程的全貌。为此,我们提出PALM(主动学习模型性能分析),一种统一且可解释的数学模型,通过四个关键参数——可达准确率、覆盖效率、早期性能和可扩展性——刻画主动学习轨迹。PALM能基于部分观测预测未来表现,支持不同策略间的严谨比较。我们在CIFAR-10/100和ImageNet-50/100/200上进行了广泛实验,涵盖多种主动学习方法与自监督嵌入。结果表明,PALM在不同数据集、预算和策略间具有强泛化能力,能从有限标注数据中准确预测完整学习曲线。更重要的是,它揭示了学习效率、数据空间覆盖及主动学习可扩展性的关键洞察。通过支持低成本策略选择与紧预算下的性能预测,PALM为研究与实际应用中的主动学习评估提供了系统化、可复现、数据高效的基准。代码已开源:https://github.com/juliamachnio/PALM。

原文摘要 · Abstract (English)

Active learning (AL) seeks to reduce annotation costs by selecting the most informative samples for labeling, making it particularly valuable in resource-constrained settings. However, traditional evaluation methods, which focus solely on final accuracy, fail to capture the full dynamics of the learning process. To address this gap, we propose PALM (Performance Analysis of Active Learning Models), a unified and interpretable mathematical model that characterizes AL trajectories through four key parameters: achievable accuracy, coverage efficiency, early-stage performance, and scalability. PALM provides a predictive description of AL behavior from partial observations, enabling the estimation of future performance and facilitating principled comparisons across different strategies. We validate PALM through extensive experiments on CIFAR-10/100 and ImageNet-50/100/200, covering a wide range of AL methods and self-supervised embeddings. Our results demonstrate that PALM generalizes effectively across datasets, budgets, and strategies, accurately predicting full learning curves from limited labeled data. Importantly, PALM reveals crucial insights into learning efficiency, data space coverage, and the scalability of AL methods. By enabling the selection of cost-effective strategies and predicting performance under tight budget constraints, PALM lays the basis for more systematic, reproducible, and data-efficient evaluation of AL in both research and real-world applications. The code is available at: https://github.com/juliamachnio/PALM.

主动学习模型评估预测效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。