用深度学习识别并屏蔽阿拉伯语社交媒体中的仇恨言论
Arabic Hate Speech Identification and Masking in Social Media using Deep Learning Models and Pre-trained Models Fine-tuning
- 用Transformer模型检测阿拉伯语仇恨言论，提升准确率
- 92%宏F1得分，文本清洗后遮蔽效果达0.3的BLEU分数
- 适合关注多语言内容安全与文本净化的研究者
社交媒体中的仇恨言论识别日益重要。本研究解决两个问题：一是识别阿拉伯语文本中的仇恨言论，二是将含有仇恨内容的文本进行净化，即用星号替换每个敏感词（星号数量等于原词字母数）。针对第一问题，采用深度学习模型和预训练Transformer进行实验，以最大化F1分数；第二问题被建模为机器翻译任务，输入为含脏话的句子，输出为已掩码的句子。最佳模型在仇恨言论检测中达到92%的宏平均F1分数和95%的准确率；在文本净化任务中，最佳掩码模型在1-gram下的BLEU得分为0.3，优于当前主流机器翻译系统表现。
原文摘要 · Abstract (English)
Hate speech identification in social media has become an increasingly important issue in recent years. In this research, we address two problems: 1) to detect hate speech in Arabic text, 2) to clean a given text from hate speech. The meaning of cleaning here is replacing each bad word with stars based on the number of letters for each word. Regarding the first problem, we conduct several experiments using deep learning models and transformers to determine the best model in terms of the F1 score. Regarding second problem, we consider it as a machine translation task, where the input is a sentence containing dirty text and the output is the same sentence with masking the dirty text. The presented methods achieve the best model in hate speech detection with a 92\% Macro F1 score and 95\% accuracy. Regarding the text cleaning experiment, the best result in the hate speech masking model reached 0.3 in BLEU score with 1-gram, which is a good result compared with the state of the art machine translation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。