用大模型生成仇恨言论净化数据集，提升自动化净化效果。
LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification
- 用大模型替代人工标注，实现自动文本净化
- 构建超8000对的仇恨言论净化数据集
- 适合研究文本净化与大模型应用者参考
随着网络有害内容激增，文本净化（Detoxification）——将有害语言重写为无害文本——变得愈发重要。然而，高质量的平行数据集，尤其是针对仇恨言论的，因人力标注成本高且敏感而稀缺。本文提出一种基于GPT-4o-mini的LLM-in-the-loop流水线，先以大模型复现ParaDetox流程，验证其性能可媲美人工标注。在此基础上，构建了大规模仇恨言论净化数据集ParaDeHate，包含超过8,000对仇恨/非仇恨文本对，并评估多种基线方法。实验表明，以ParaDeHate微调的BART模型在风格准确性、内容保留度和流畅性上表现更优，证明大模型生成的净化文本可作为人工作标注的可扩展替代方案。
原文摘要 · Abstract (English)
Detoxification, the task of rewriting harmful language into non-toxic text, has become increasingly important amid the growing prevalence of toxic content online. However, high-quality parallel datasets for detoxification, especially for hate speech, remain scarce due to the cost and sensitivity of human annotation. In this paper, we propose a novel LLM-in-the-loop pipeline leveraging GPT-4o-mini for automated detoxification. We first replicate the ParaDetox pipeline by replacing human annotators with an LLM and show that the LLM performs comparably to human annotation. Building on this, we construct ParaDeHate, a large-scale parallel dataset specifically for hatespeech detoxification. We release ParaDeHate as a benchmark of over 8K hate/non-hate text pairs and evaluate a wide range of baseline methods. Experimental results show that models such as BART, fine-tuned on ParaDeHate, achieve better performance in style accuracy, content preservation, and fluency, demonstrating the effectiveness of LLM-generated detoxification text as a scalable alternative to human annotation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。