多语言文本净化新方法，可解释且支持5种新语言
Multilingual and Explainable Text Detoxification with Parallel Corpora
- 基于平行语料库，构建跨语言净化模型
- 首次实现9语言毒性特征的自动化可解释分析
- 引入思维链思想提升提示效果，适合多语言安全应用
尽管各国及社交媒体平台已出台多项监管政策（印度政府，2021；欧盟议会与理事会，2022），网络暴力言论仍为重大挑战。自动文本净化作为文本风格迁移的一种，旨在将有毒语言转化为更中性或非毒性的表达。现有先进方法依赖于平行语料库，本研究将文本净化语料库扩展至德语、中文、阿拉伯语、印地语和阿姆哈拉语五种新语言，并在多语言设置下测试了多种文本风格迁移基线模型。此外，首次开展自动化、可解释的毒性与非毒性句子描述特征分析，深入探讨了9种语言中毒性与净化的细微差别。最后，基于上述发现，提出一种受思维链推理启发的新文本净化方法，通过聚类相关描述属性优化提示过程。
原文摘要 · Abstract (English)
Even with various regulations in place across countries and social media platforms (Government of India, 2021; European Parliament and Council of the European Union, 2022, digital abusive speech remains a significant issue. One potential approach to address this challenge is automatic text detoxification, a text style transfer (TST) approach that transforms toxic language into a more neutral or non-toxic form. To date, the availability of parallel corpora for the text detoxification task (Logachevavet al., 2022; Atwell et al., 2022; Dementievavet al., 2024a) has proven to be crucial for state-of-the-art approaches. With this work, we extend parallel text detoxification corpus to new languages -- German, Chinese, Arabic, Hindi, and Amharic -- testing in the extensive multilingual setup TST baselines. Next, we conduct the first of its kind an automated, explainable analysis of the descriptive features of both toxic and non-toxic sentences, diving deeply into the nuances, similarities, and differences of toxicity and detoxification across 9 languages. Finally, based on the obtained insights, we experiment with a novel text detoxification method inspired by the Chain-of-Thoughts reasoning approach, enhancing the prompting process through clustering on relevant descriptive attributes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。