用英语微调可降低多语言模型的偏见与毒性，但会影响非英语生成质量。
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation
- 通过英文无害文本微调，提升多语言模型的去偏与去毒能力。
- 仅直接偏好优化能有效降低毒性，且效果可跨语言迁移。
- 迁移效果取决于预训练数据中该语言的占比，需警惕生成能力下降。
近期生成式大语言模型在非英语语言上表现优异，但提示时仍易产生较高社会偏见和毒性内容。已有研究表明，在英文数据上微调可缓解此问题，并实现向其他语言的迁移。本文系统研究了不同微调方法对模型偏见、毒性及生成流畅性与多样性的影响。通过使用精选无害文本微调可减少偏见，但仅直接偏好优化能有效降低毒性。在英语中实施的缓解措施可迁移到非英语语言，迁移程度与模型预训练数据中该语言的占比相关。然而，这种迁移常伴随非英语生成能力下降，凸显开发语言特异性去偏去毒方法的重要性。
原文摘要 · Abstract (English)
Recent generative large language models (LLMs) show remarkable performance in non-English languages, but when prompted in those languages they tend to express higher harmful social biases and toxicity levels. Prior work has shown that finetuning on specialized datasets can mitigate this behavior, and doing so in English can transfer to other languages. In this work, we investigate the impact of different finetuning methods on the model's bias and toxicity, but also on its ability to produce fluent and diverse text. We reduce biases by finetuning on curated non-harmful text, but find only direct preference optimization to be effective for mitigating toxicity. The mitigation caused by applying these methods in English also transfers to non-English languages. We find evidence that the extent to which transfer takes place can be predicted by the amount of data in a given language present in the model's pretraining data. However, this transfer of bias and toxicity mitigation often comes at the expense of decreased language generation ability in non-English languages, highlighting the importance of developing language-specific bias and toxicity mitigation methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。