针对印度低资源语言的毒害内容检测框架，实现84.23%平均准确率
UNITYAI-GUARD: Pioneering Toxicity Detection Across Low-Resource Indian Languages
- 构建多语言毒性分类模型，支持七种印度语文字
- 在56.7万训练样本上达到84.23%平均F1分数
- 开源API助力跨领域应用，推动本地化内容安全
本文提出UnityAI-Guard，一种面向低资源印度语言的二元毒害内容分类框架。现有系统多聚焦高资源语言，而该框架填补了这一空白，针对多种婆罗米系/印地语文字开发了先进模型。实验在包含56.7万训练样本和3万条人工标注测试样本的数据集上进行，跨七种语言平均F1得分达84.23%。通过提升多语言内容审核能力，本工作还提供公开API接口，促进技术在语言多样性区域的广泛应用。
原文摘要 · Abstract (English)
This work introduces UnityAI-Guard, a framework for binary toxicity classification targeting low-resource Indian languages. While existing systems predominantly cater to high-resource languages, UnityAI-Guard addresses this critical gap by developing state-of-the-art models for identifying toxic content across diverse Brahmic/Indic scripts. Our approach achieves an impressive average F1-score of 84.23% across seven languages, leveraging a dataset of 567k training instances and 30k manually verified test instances. By advancing multilingual content moderation for linguistically diverse regions, UnityAI-Guard also provides public API access to foster broader adoption and application.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。