用大模型自动验证文本分类结果，降低人工标注成本。
Large Language Models in the Task of Automatic Validation of Text Classifier Predictions
- 用大模型替代人工标注，自动检验分类器预测是否正确。
- 可有效应对数据漂移问题，支持模型持续迭代。
- 适合需要频繁更新的文本分类系统，节省人力成本。
文本分类模型需依赖人工标注数据进行训练与验证，但获取高质量标注需投入大量人力，且专家资源有限、效率受制于人。尽管一次性收集数据成本尚可接受，但为应对数据漂移（模型漂移）而频繁重训模型时，重复标注过程成为长期且高昂的负担。本文提出多种基于大语言模型（LLMs）的方法，用于自动验证分类器预测结果的正确性，从而保障模型质量，并支撑高质量的增量学习流程。
原文摘要 · Abstract (English)
Machine learning models for text classification are trained to predict a class for a given text. To do this, training and validation samples must be prepared: a set of texts is collected, and each text is assigned a class. These classes are usually assigned by human annotators with different expertise levels, depending on the specific classification task. Collecting such samples from scratch is labor-intensive because it requires finding specialists and compensating them for their work; moreover, the number of available specialists is limited, and their productivity is constrained by human factors. While it may not be too resource-intensive to collect samples once, the ongoing need to retrain models (especially in incremental learning pipelines) to address data drift (also called model drift) makes the data collection process crucial and costly over the model's entire lifecycle. This paper proposes several approaches to replace human annotators with Large Language Models (LLMs) to test classifier predictions for correctness, helping ensure model quality and support high-quality incremental learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。