针对法语毒性内容检测,构建了5万+样本数据集并提出新训练方法。
ToxiFrench: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detection
- 用大模型预标注+人工校验,仅10%人力完成53,622条法语评论标注
- 小模型反而比大模型更抗干扰,4B模型经提示链微调后准确率提升10%
- 适合关注多语言安全、模型可解释性与高效训练的研究者
使用语言模型检测毒性内容至关重要但极具挑战。尽管英语领域进展显著,法语文本的毒性检测仍发展滞后,主要因缺乏文化相关、人工标注且规模可观的数据集。本文发布ToxiFrench数据集,包含53,622条法语在线评论,并提供平衡的基准划分以实现系统评估。该数据集通过半自动标注流程构建,利用高置信度大模型预标注,将人工标注工作量降至10%,同时确保统计上与纯人工标注一致。我们对多种模型进行基准测试,发现反直觉现象:小语言模型(SLMs)在鲁棒性和泛化能力上常优于大型模型。基于此,提出一种新型链式思维(CoT)微调策略,结合动态加权损失(DWL),逐步强化模型最终决策,显著提升判断忠实度。经微调的4B模型(Qwen3-4B)在基准上达到最先进水平,其平衡准确率较基线提升10%,性能优于GPT-4o和DeepSeek-R1,且保持跨语言能力。
原文摘要 · Abstract (English)
Detecting toxic content using language models is crucial yet challenging. While substantial progress has been made in English, toxicity detection in French remains underdeveloped, primarily due to the lack of culturally relevant, human-annotated, large-scale datasets. In this work, we release ToxiFrench, a dataset of 53,622 French online comments together with a balanced benchmark split for systematic evaluation. The dataset is constructed via a semi-automated annotation pipeline that reduces manual labeling to only 10% through high-confidence LLM-based pre-annotation and human verification, while ensuring statistical alignment with human-only annotation. We then benchmark a broad range of models and uncover a counterintuitive finding: Small Language Models (SLMs) often surpass larger models in robustness and generalization on this task. Motivated by this finding, we propose a novel Chain-of-Thought (CoT) fine-tuning strategy using a Dynamic Weighted Loss (DWL) that progressively emphasizes the model's final decision and significantly improves faithfulness. Our fine-tuned 4B model (Qwen3-4B) achieves state-of-the-art performance on the benchmark. It improves its balanced accuracy by 10% over its baseline and achieves better performance than GPT-4o and DeepSeek-R1 on our benchmark, while successfully retaining cross-lingual capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。