arXiv:2505.12116cs.CL2025-05NeurIPS被引 2

构建首个提格雷尼亚语反语言检测多任务基准,助力低资源语言内容安全

A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings

  • 基于9位母语者标注1.3万条推特评论,支持罗马字母与盖兹字母双书写系统
  • 小模型微调在反语言检测上达86.67% F1,超越大模型7个百分点
  • 适合关注低资源语言、在线安全与多任务学习的研究者使用

内容审核研究虽有进展,却因资源匮乏难以覆盖全球多数语言,使数百万用户暴露于网络暴力中。本文构建了一个大规模人工标注的多任务基准数据集,用于提格雷尼亚语社交媒体中的反语言检测,包含三个联合标注任务:反语言性、情感和主题分类。数据集涵盖13,717条来自7,373个视频的YouTube评论,涉及51个频道,总播放量超12亿次,由九位母语者标注。针对约64%的提格雷尼亚语内容使用罗马化拼写而非本地盖兹文字的特点,数据集同时支持两种书写系统。我们采用迭代术语聚类方法进行有效数据筛选,并建立各任务强基线。实验表明,在低资源环境下,小型微调模型优于提示式大语言模型(LLM),在反语言检测任务中达到86.67% F1,优于最佳LLM超过7个百分点,并在其余任务中保持更强性能。该基准已公开,以推动在线安全研究。

原文摘要 · Abstract (English)

Content moderation research has recently made significant advances, but remains limited in serving the majority of the world's languages due to the lack of resources, leaving millions of vulnerable users to online hostility. This work presents a large-scale human-annotated multi-task benchmark dataset for abusive language detection in Tigrinya social media with joint annotations for three tasks: abusiveness, sentiment, and topic classification. The dataset comprises 13,717 YouTube comments annotated by nine native speakers, collected from 7,373 videos with a total of over 1.2 billion views across 51 channels. We developed an iterative term clustering approach for effective data selection. Recognizing that around 64% of Tigrinya social media content uses Romanized transliterations rather than native Ge'ez script, our dataset accommodates both writing systems to reflect actual language use. We establish strong baselines across the tasks in the benchmark, while leaving significant challenges for future contributions. Our experiments demonstrate that small fine-tuned models outperform prompted frontier large language models (LLMs) in the low-resource setting, achieving 86.67% F1 in abusiveness detection (7+ points over best LLM), and maintain stronger performance in all other tasks. The benchmark is made public to promote research on online safety.

反语言检测低资源语言多任务学习内容安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。