通过添加惩罚项，让多语言大模型生成更专注的单一语言回答。
Controlling Language Confusion in Multilingual LLMs
- 在标准微调基础上加入惩罚项，抑制跨语言混杂输出。
- 即使高温解码下仍保持语言一致性，且不影响问答性能。
- 特别适合低资源语言场景，提升实际使用体验。
大型语言模型常出现语言混淆问题，即生成内容部分或全部为非预期语言，严重影响用户体验，尤其在低资源语境下。我们假设此问题源于传统微调目标（如监督学习）的局限性——仅优化正确标记的可能性，未显式惩罚错误输出，如跨语言混合。对预训练过程中的损失轨迹分析显示，模型无法区分单语与混合语言文本，表明缺乏内在机制避免此类混淆。本文采用ORPO，在标准SFT基础上增加对不想要输出风格的惩罚，有效抑制语言混淆生成。该方法在高解码温度下仍能保持强语言一致性，同时维持良好的问答性能。研究结果表明，引入适当的惩罚项可有效缓解多语言模型的语言混淆问题，尤其在低资源场景中效果显著。
原文摘要 · Abstract (English)
Large language models often suffer from language confusion, a phenomenon in which responses are partially or entirely generated in unintended languages. This critically degrades the user experience, especially in low-resource settings. We hypothesize that this issue stems from limitations in conventional fine-tuning objectives, such as supervised learning, which optimize the likelihood of correct tokens without explicitly penalizing undesired outputs such as cross-lingual mixing. Analysis of loss trajectories during pretraining further reveals that models fail to distinguish between monolingual and language-mixed texts, highlighting the absence of inherent pressure to avoid such confusion. In this work, we apply ORPO, which adds penalties for unwanted output styles to standard SFT, effectively suppressing language-confused generations. ORPO maintains strong language consistency, even under high decoding temperatures, while preserving general QA performance. Our findings suggest that incorporating appropriate penalty terms can effectively mitigate language confusion in multilingual models, particularly in low-resource scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。