ATGen让文本生成任务用主动学习减少标注成本。
ATGen: A Framework for Active Text Generation
- 融合人工与大模型自动标注，支持云端和本地LLM
- 在多个生成任务中降低人力与API调用成本
- 专为文本生成设计的主动学习框架，适合研究者使用
主动学习（AL）在减少机器学习模型训练标注成本方面展现出巨大潜力。然而，尽管近年来自然语言生成（NLG）任务日益流行，主动学习在该领域的应用仍有限。本文提出主动文本生成（ATGen）——一个全面的框架，将主动学习与文本生成任务相结合，使最先进的主动学习策略可应用于文本生成。该框架通过人类标注员和基于大语言模型（LLM）的自动标注代理，简化了主动学习驱动的标注流程，支持部署为服务的LLM（如ChatGPT、Claude）或本地运行的模型。此外，ATGen提供统一平台，便于实现和基准测试专为NLG任务设计的新主动学习策略。我们对多种设置下主流主动学习策略在多个文本生成任务上的表现进行了评估，结果表明，ATGen显著降低了人工标注负担及基于LLM的标注代理的API调用成本。框架代码已在GitHub以MIT许可证开源，视频演示见http://atgen-video.nlpresearch.group。
原文摘要 · Abstract (English)
Active learning (AL) has demonstrated remarkable potential in reducing the annotation effort required for training machine learning models. However, despite the surging popularity of natural language generation (NLG) tasks in recent years, the application of AL to NLG has been limited. In this paper, we introduce Active Text Generation (ATGen) - a comprehensive framework that bridges AL with text generation tasks, enabling the application of state-of-the-art AL strategies to NLG. Our framework simplifies AL-empowered annotation in NLG tasks using both human annotators and automatic annotation agents based on large language models (LLMs). The framework supports LLMs deployed as services, such as ChatGPT and Claude, or operated on-premises. Furthermore, ATGen provides a unified platform for smooth implementation and benchmarking of novel AL strategies tailored to NLG tasks. Finally, we present evaluation results for state-of-the-art AL strategies across diverse settings and multiple text generation tasks. We show that ATGen reduces both the effort of human annotators and costs associated with API calls to LLM-based annotation agents. The code of the framework is available on GitHub under the MIT license. The video presentation is available at http://atgen-video.nlpresearch.group
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。