arXiv:2507.01050cs.LGcs.AI2025-07EMNLP被引 1

用少量标注数据训练文本去毒模型,兼顾语义保留与泛化能力。

Text Detoxification: Data Efficiency, Semantic Preservation and Model Generalization

  • 两阶段训练:先小数据微调,再用无标注毒文+自定义奖励模型优化
  • 在不依赖大量标注数据情况下,实现顶尖去毒性能和跨域泛化
  • 适合需要低成本、高鲁棒性的内容安全系统开发者

社交媒体中毒性内容的广泛传播对网络环境和公共讨论构成严重威胁,亟需既能有效去毒又保留原语义的方法。现有方法常难以同时实现强去毒效果、语义保真和对分布外数据的鲁棒性,且多依赖昂贵的人工标注平行语料,数据效率低下。为此,我们提出一种两阶段训练框架,联合优化数据效率、语义保真与模型泛化能力。首先在少量高质量过滤后的平行数据上进行监督微调,获得良好初始化;随后利用未标注的毒性输入和自定义奖励模型,通过组相对策略优化(Group Relative Policy Optimization)训练大语言模型。实验表明,该方法有效缓解了以往工作中的权衡问题,在显著降低对标注数据依赖的同时,达到领先性能并提升泛化能力。代码已公开于 https://github.com/allacnobug/Detoxification-of-Text。

原文摘要 · Abstract (English)

The widespread dissemination of toxic content on social media poses a serious threat to both online environments and public discourse, highlighting the urgent need for detoxification methods that effectively remove toxicity while preserving the original semantics. However, existing approaches often struggle to simultaneously achieve strong detoxification performance, semantic preservation, and robustness to out-of-distribution data. Moreover, they typically rely on costly, manually annotated parallel corpora while showing poor data efficiency. To address these challenges, we propose a two-stage training framework that jointly optimizes for data efficiency, semantic preservation, and model generalization. We first perform supervised fine-tuning on a small set of high-quality, filtered parallel data to establish a strong initialization. Then, we leverage unlabeled toxic inputs and a custom-designed reward model to train the LLM using Group Relative Policy Optimization. Experimental results demonstrate that our method effectively mitigates the trade-offs faced by previous work, achieving state-of-the-art performance with improved generalization and significantly reduced dependence on annotated data. Our code is available at: https://github.com/allacnobug/Detoxification-of-Text.

文本去毒数据效率语义保真

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。