用大模型+翻译方法,提升多语言仇恨言论检测效果。
Multilingual Hate Speech Detection in Social Media Using Translation-Based Approaches with Large Language Models
- 结合注意力层与大模型,增强跨语言特征提取能力。
- 在英、西、乌三语上实现最高0.88的宏F1分数。
- 首个针对乌尔都语的三语仇恨言论数据集,适合安全研究者使用。
社交媒体是公众讨论的重要空间,但其广泛使用也加剧了有害内容,尤其是仇恨言论,威胁网络环境安全与包容性。尽管英语和西班牙语的仇恨言论检测已受广泛关注,乌尔都语仍缺乏系统研究,尤其在基于翻译的方法方面。为此,本文构建了一个包含10,193条推文的三语数据集(英语3,834条,乌尔都语3,197条,西班牙语3,162条),通过关键词筛选获得,仇恨与非仇恨标签各4,849和5,344条,分布均衡。采用注意力层预处理后接入基于Transformer的模型与大语言模型(LLMs)进行特征提取;非Transformer模型则使用TF-IDF。数据集由三位标注员依据严格标准标注,一致性达Fleiss' Kappa 0.821。实验表明,结合注意力层与GPT-3.5 Turbo及Qwen 2.5 72B模型,在英语(0.87)、西班牙语(0.85)、乌尔都语(0.81)及联合多语言模型(0.88)上均表现优异,较SVM基线分别提升8.75%、8.97%、5.19%和7.32%,验证了该框架在多语言仇恨言论检测中的有效性。
原文摘要 · Abstract (English)
Social media platforms are critical spaces for public discourse, shaping opinions and community dynamics, yet their widespread use has amplified harmful content, particularly hate speech, threatening online safety and inclusivity. While hate speech detection has been extensively studied in languages like English and Spanish, Urdu remains underexplored, especially using translation-based approaches. To address this gap, we introduce a trilingual dataset of 10,193 tweets in English (3,834 samples), Urdu (3,197 samples), and Spanish (3,162 samples), collected via keyword filtering, with a balanced distribution of 4,849 Hateful and 5,344 Not-Hateful labels. Our methodology leverages attention layers as a precursor to transformer-based models and large language models (LLMs), enhancing feature extraction for multilingual hate speech detection. For non-transformer models, we use TF-IDF for feature extraction. The dataset is benchmarked using state-of-the-art models, including GPT-3.5 Turbo and Qwen 2.5 72B, alongside traditional machine learning models like SVM and other transformers (e.g., BERT, RoBERTa). Three annotators, following rigorous guidelines, ensured high dataset quality, achieving a Fleiss' Kappa of 0.821. Our approach, integrating attention layers with GPT-3.5 Turbo and Qwen 2.5 72B, achieves strong performance, with macro F1 scores of 0.87 for English (GPT-3.5 Turbo), 0.85 for Spanish (GPT-3.5 Turbo), 0.81 for Urdu (Qwen 2.5 72B), and 0.88 for the joint multilingual model (Qwen 2.5 72B). These results reflect improvements of 8.75% in English (over SVM baseline 0.80), 8.97% in Spanish (over SVM baseline 0.78), 5.19% in Urdu (over SVM baseline 0.77), and 7.32% in the joint multilingual model (over SVM baseline 0.82). Our framework offers a robust solution for multilingual hate speech detection, fostering safer digital communities worldwide.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。