调研NLP社区主动学习应用现状,发现三大难题仍存。
Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey
- 通过在线调查收集社区实践数据,分析主动学习落地难点。
- 80%以上受访者认为标注重要性持续,主动学习仍具价值。
- 针对工具缺失、成本不确定等问题提出简化方案。
监督学习依赖人工标注,通常耗时且昂贵。主动学习作为一种迭代策略,仅让人类标注模型认为有信息量的数据实例,可降低标注成本。尽管大语言模型(LLMs)推动了主动学习研究进展,但其在真实场景中的应用情况仍不明确。为此,我们对NLP社区开展在线调查,收集关于当前实施实践、应用障碍及未来前景的非量化洞察。结果表明:数据标注预计仍具重要性,主动学习也持续相关,并将受益于LLMs。与15年前的同类调查一致,三大核心挑战依然存在——配置复杂、成本降低不确定、工具支持不足。本文提出缓解策略,并发布匿名化数据集。
原文摘要 · Abstract (English)
Supervised learning relies on data annotation which usually is time-consuming and therefore expensive. A longstanding strategy to reduce annotation costs is active learning, an iterative process, in which a human annotates only data instances deemed informative by a model. Research in active learning has made considerable progress, especially with the rise of large language models (LLMs). However, we still know little about how these remarkable advances have translated into real-world applications, or contributed to removing key barriers to active learning adoption. To fill in this gap, we conduct an online survey in the NLP community to collect previously intangible insights on current implementation practices, common obstacles in application, and future prospects in active learning. We also reassess the perceived relevance of data annotation and active learning as fundamental assumptions. Our findings show that data annotation is expected to remain important and active learning to stay relevant while benefiting from LLMs. Consistent with a community survey from over 15 years ago, three key challenges yet persist -- setup complexity, uncertain cost reduction, and tooling -- for which we propose alleviation strategies. We publish an anonymized version of the dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。