arXiv:2411.19638cs.CL2024-11中稿 · and published in t…被引 20

用大模型自动标注新闻,零人工成本实现多语言主题分类。

LLM Teacher-Student Framework for Text Classification With No Manually Annotated Data: A Case Study in IPTC News Topic Classification

  • 用GPT做教师模型,自动为2万篇多语种新闻打标签。
  • 小模型学生版在少量数据下达到接近教师模型的准确率。
  • 支持跨语言零样本分类,适合多语言新闻系统部署。

随着网络新闻数量激增,跨语言主题分类对提升读者内容获取效率至关重要。本文提出一种基于大语言模型(LLM)的师生框架,无需人工标注即可构建轻量级多语言新闻主题分类模型。该框架使用GPT作为教师模型,自动标注4种语言(斯洛文尼亚语、克罗地亚语、希腊语、加泰罗尼亚语)共20,000篇新闻文章,归类至国际新闻电讯委员会(IPTC)媒体主题架构中的17个主类别。教师模型在所有语言中均表现出高零样本性能,其标注一致性与人工标注者之间相当。为应对大规模文本处理的计算瓶颈,采用小型BERT-like学生模型在GPT标注数据集上微调。学生模型性能接近教师模型。研究还发现,学生模型在较少训练样本下即达高精度,并展现出强零样本跨语言能力。最终发布最佳分类器,支持基于IPTC媒体主题架构的多语言主题分类。

原文摘要 · Abstract (English)

With the ever-increasing number of news stories available online, classifying them by topic, regardless of the language they are written in, has become crucial for enhancing readers' access to relevant content. To address this challenge, we propose a teacher-student framework based on large language models (LLMs) for developing multilingual news topic classification models of reasonable size with no need for manual data annotation. The framework employs a Generative Pretrained Transformer (GPT) model as the teacher model to develop a news topic training dataset through automatic annotation of 20,000 news articles in Slovenian, Croatian, Greek, and Catalan. Articles are classified into 17 main categories from the Media Topic schema, developed by the International Press Telecommunications Council (IPTC). The teacher model exhibits high zero-shot performance in all four languages. Its agreement with human annotators is comparable to that between the human annotators themselves. To mitigate the computational limitations associated with the requirement of processing millions of texts daily, smaller BERT-like student models are fine-tuned on the GPT-annotated dataset. These student models achieve high performance comparable to the teacher model. Furthermore, we explore the impact of the training data size on the performance of the student models and investigate their monolingual, multilingual, and zero-shot cross-lingual capabilities. The findings indicate that student models can achieve high performance with a relatively small number of training instances, and demonstrate strong zero-shot cross-lingual abilities. Finally, we publish the best-performing news topic classifier, enabling multilingual classification with the top-level categories of the IPTC Media Topic schema.

文本分类大模型零样本多语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。