用网络数据和大模型标注，提升多语言仇恨言论检测效果
Toward Generalized Cross-Lingual Hateful Language Detection with Web-Scale Data and Ensemble LLM Annotations
- 用网页未标注数据继续预训练BERT，提升低资源语言表现
- 用多个大模型加权投票生成合成标签，小模型性能提升11%
- 对小模型和低资源语言最有效，适合资源有限的场景
我们研究大规模未标注网络数据与基于大模型的合成标注是否能提升多语言仇恨言论检测。从四个语言（英语、德语、西班牙语、越南语）的OpenWebSearch.eu（OWS）爬取文本，采用两种互补策略：首先，在未标注的OWS文本上继续进行掩码语言建模，对BERT模型进行持续预训练，再进行监督微调，结果显示在十六个基准测试中平均宏F1提升约3%，低资源场景下增益更显著；其次，使用四个开源大模型（Mistral-7B、Llama3.1-8B、Gemma2-9B、Qwen2.5-14B）通过均值平均、多数投票和LightGBM元学习器三种集成策略生成合成标注，其中LightGBM集成表现最佳。用这些合成标签微调小模型（Llama3.2-1B）时，整体F1提升11%，而对大模型Qwen2.5-14B仅提升0.6%。结果表明，结合网络规模数据与大模型集成标注，对小模型和低资源语言最具价值。
原文摘要 · Abstract (English)
We study whether large-scale unlabelled web data and LLM-based synthetic annotations can improve multilingual hate speech detection. Starting from texts crawled via OpenWebSearch.eu~(OWS) in four languages (English, German, Spanish, Vietnamese), we pursue two complementary strategies. First, we apply continued pre-training to BERT models by continuing masked language modelling on unlabelled OWS texts before supervised fine-tuning, and show that this yields an average macro-F1 gain of approximately 3% over standard baselines across sixteen benchmarks, with stronger gains in low-resource settings. Second, we use four open-source LLMs (Mistral-7B, Llama3.1-8B, Gemma2-9B, Qwen2.5-14B) to produce synthetic annotations through three ensemble strategies: mean averaging, majority voting, and a LightGBM meta-learner. The LightGBM ensemble consistently outperforms the other strategies. Fine-tuning on these synthetic labels substantially benefits a small model (Llama3.2-1B: +11% pooled F1), but provides only a modest gain for the larger Qwen2.5-14B (+0.6%). Our results indicate that the combination of web-scale unlabelled data and LLM-ensemble annotations is the most valuable for smaller models and low-resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。