对比大模型与传统方法在文本分类中的表现，发现大模型更擅长复杂任务但推理慢。
Large Language Models For Text Classification: Case Study And Comprehensive Review
- 对比不同大小、量化和架构的大模型在两类分类任务中的表现
- Llama3和GPT-4在多分类任务上超越传统方法，但推理时间长
- 简单机器学习模型在二分类任务中性能与速度更优，适合实时场景
探索大语言模型（LLMs）在文本分类中的潜力是自然语言处理的重要方向。本文评估了多种大模型在两种分类场景下的表现：基于在线职位评论的员工工作地点多分类，以及新闻真伪二分类。模型涵盖不同规模、量化方式和架构，对比其在加权F1分数上的表现，并考察提示策略的影响。同时分析各模型在性能（F1-score）与推理时间之间的权衡，以评估实际应用价值。结果表明，提示策略显著影响模型输出；在复杂多分类任务中，尤其是使用Llama3和GPT-4时，大模型表现优于传统深度学习与机器学习方法，但推理耗时更长。而在简单的二分类任务中，传统模型展现出更高的性能-时间比，更适合高效部署。
原文摘要 · Abstract (English)
Unlocking the potential of Large Language Models (LLMs) in data classification represents a promising frontier in natural language processing. In this work, we evaluate the performance of different LLMs in comparison with state-of-the-art deep-learning and machine-learning models, in two different classification scenarios: i) the classification of employees' working locations based on job reviews posted online (multiclass classification), and 2) the classification of news articles as fake or not (binary classification). Our analysis encompasses a diverse range of language models differentiating in size, quantization, and architecture. We explore the impact of alternative prompting techniques and evaluate the models based on the weighted F1-score. Also, we examine the trade-off between performance (F1-score) and time (inference response time) for each language model to provide a more nuanced understanding of each model's practical applicability. Our work reveals significant variations in model responses based on the prompting strategies. We find that LLMs, particularly Llama3 and GPT-4, can outperform traditional methods in complex classification tasks, such as multiclass classification, though at the cost of longer inference times. In contrast, simpler ML models offer better performance-to-time trade-offs in simpler binary classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。