用高效微调方法提升低资源语言仇恨言论检测效果
Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu
- 采用低秩自适应（LoRA）仅微调少量参数
- 在7.2万条数据上实现超过0.93的F1分数
- 适合资源有限但需高精度文本分类的场景
低资源语言（LRLs）的仇恨言论检测面临标注数据匮乏、语言结构非正式和语法不规范等挑战。以南亚社交媒体中广泛使用的罗马乌尔都语为例，其拼写不一致且上下文依赖强。本文对Mistral、LLaMA、Falcon及多语言BERT等多种Transformer模型，在包含72,000条标注评论的PURUTT数据集上，评估了零样本推理与参数高效微调（PEFT）方法——低秩自适应（LoRA）的效果。结果显示，零样本模型表现中等（F1=0.56），而仅微调少量可训练参数即显著提升性能（F1 > 0.93）。PEFT不仅取得优异准确率，还具备极佳计算效率，非常适合低资源语言处理任务。
原文摘要 · Abstract (English)
It is challenging to detect hate speech in Low Resource Languages (LRLs) because of the absence of annotated data, the informality of its language structure, and the lack of standardized grammar. A good example of such a challenge is Roman Urdu which is broadly used by South Asians on social media and has a high variation while lacking contextually consistent spellings. The objective of this paper is to conduct a comprehensive assessment of Large Language Models (LLMs) for Hate Speech Detection (HSD) in Roman Urdu script and fine-tune these models using the Parameter-Efficient Fine-Tuning (PEFT) method called Low-Rank Adaptation (LoRA). To evaluate zero-shot inference, we benchmarked it against PEFT on different transformer models, including Mistral, LLaMA, Falcon, and multilingual BERT. Experiments are conducted on the PURUTT (Parallel Urdu and Roman Urdu Corpus for Toxic Comments and Transliteration) dataset with over 72,000 annotated comments. The results suggest that zero shot models perform moderately (F1 = 0.56), but updating a small fraction of the model trainable parameters improves the classification performance significantly (F1 > 0.93). Our results have shown that PEFT delivers outstanding performance alongside excellent computational efficiency, making it highly suitable for low-resource language processing tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。