用大模型辅助主动学习，大幅降低阿拉伯语情感分析的标注成本。
From Human Annotation to Automation: LLM-in-the-Loop Active Learning for Arabic Sentiment Analysis
- 引入大模型作为标注助手，结合主动学习减少人工标注量。
- 在三个数据集上，仅用450-650条标注样本即达93%和82%准确率。
- 适合需要低成本高效率构建阿拉伯语情感分析系统的团队使用。
自然语言处理中的情感分析在营销、客服和社交媒体监控中至关重要，但阿拉伯语情感分析因缺乏大规模高质量标注数据而进展缓慢。主动学习虽可减少标注成本，但在阿拉伯语任务中研究较少；大模型辅助标注在阿拉伯语领域也尚未充分探索。本文提出一种面向阿拉伯语情感分析的主动学习框架，评估了LSTM、GRU和RNN三种深度学习模型在Hunger Station、AJGT和MASAC三个基准数据集上的表现，涵盖现代标准阿拉伯语与方言。对比了人工标注与大模型辅助标注两种策略，测试了GPT-4o、Claude 3 Sonnet、Gemini 2.5 Pro、DeepSeek Chat和LLaMA 3 70B Instruct五种大模型作为标注器的表现。每个数据集选用最优大模型：Hunger Station用GPT-4o，AJGT用Claude 3 Sonnet，MASAC用DeepSeek Chat。结果表明，大模型辅助的主动学习在性能上可媲美甚至超越人工标注：在Hunger Station上，使用GPT-4o标注的LSTM模型仅需450个样本即达93%准确率；在MASAC上，使用DeepSeek Chat标注的模型以650个样本达到82%准确率，与人工标注效果相当。
原文摘要 · Abstract (English)
Natural language processing (NLP), particularly sentiment analysis, plays a vital role in areas like marketing, customer service, and social media monitoring by providing insights into user opinions and emotions. However, progress in Arabic sentiment analysis remains limited due to the lack of large, high-quality labeled datasets. While active learning has proven effective in reducing annotation efforts in other languages, few studies have explored it in Arabic sentiment tasks. Likewise, the use of large language models (LLMs) for assisting annotation and comparing their performance to human labeling is still largely unexplored in the Arabic context. In this paper, we propose an active learning framework for Arabic sentiment analysis designed to reduce annotation costs while maintaining high performance. We evaluate multiple deep learning architectures: Specifically, long short-term memory (LSTM), gated recurrent units (GRU), and recurrent neural networks (RNN), across three benchmark datasets: Hunger Station, AJGT, and MASAC, encompassing both modern standard Arabic and dialectal variations. Additionally, two annotation strategies are compared: Human labeling and LLM-assisted labeling. Five LLMs are evaluated as annotators: GPT-4o, Claude 3 Sonnet, Gemini 2.5 Pro, DeepSeek Chat, and LLaMA 3 70B Instruct. For each dataset, the best-performing LLM was used: GPT-4o for Hunger Station, Claude 3 Sonnet for AJGT, and DeepSeek Chat for MASAC. Our results show that LLM-assisted active learning achieves competitive or superior performance compared to human labeling. For example, on the Hunger Station dataset, the LSTM model achieved 93% accuracy with only 450 labeled samples using GPT-4o-generated labels, while on the MASAC dataset, DeepSeek Chat reached 82% accuracy with 650 labeled samples, matching the accuracy obtained through human labeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。