用协作式提示让大模型更准地标注文本。
Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting
- 让多个大模型通过推理过程协作，提升标注一致性。
- 在4个数据集上，协作方法优于传统少样本提示。
- 适合需要高精度标注的复杂文本任务使用。
传统数据标注过程往往耗时耗力，易受人为偏见影响，给日益复杂的数据集管理带来挑战。本研究探索利用大语言模型（LLMs）作为自动化标注工具，以提高标注效率与一致性。通过采用基于推理过程的协作式少样本提示技术，我们旨在提升LLMs在文本标注任务中的表现。在四个基准数据集上对六种LLM进行了严格评估，比较了七种不同方法。结果表明，协作方法在复杂标注任务中始终优于传统少样本技术及其他基线方法。本工作为利用协作学习方法应对挑战性文本标注任务提供了宝贵见解和稳健框架。
原文摘要 · Abstract (English)
The traditional data annotation process is often labor-intensive, time-consuming, and susceptible to human bias, which complicates the management of increasingly complex datasets. This study explores the potential of large language models (LLMs) as automated data annotators to improve efficiency and consistency in annotation tasks. By employing rationale-driven collaborative few-shot prompting techniques, we aim to improve the performance of LLMs in text annotation. We conduct a rigorous evaluation of six LLMs across four benchmark datasets, comparing seven distinct methodologies. Our results demonstrate that collaborative methods consistently outperform traditional few-shot techniques and other baseline approaches, particularly in complex annotation tasks. Our work provides valuable insights and a robust framework for leveraging collaborative learning methods to tackle challenging text annotation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。