轻量级后处理方法，显著降低多语言大模型的母语偏见。
Smoothie-Qwen: Post-Hoc Smoothing to Reduce Language Bias in Multilingual LLMs
- 通过调整词元级概率，抑制非目标语言输出
- 在Qwen上使中文误生成减少超95%，任务准确率不变
- 无需重训练，适合快速提升多语言可控性
多语言大模型常出现语言混淆问题，即无论提示语言如何，倾向于用主导语言生成回应。为此，我们提出Smoothie-Qwen，一种轻量级、无需重训练的后处理方法，通过选择性调节词元级输出概率，有效抑制非期望语言生成。应用于Qwen模型时，该方法使非意图中文输出减少超过95%，同时在多语言基准测试中保持任务准确率。本工作为提升大模型的语言可控性提供了高效实用的解决方案，使其更适用于全球应用场景。
原文摘要 · Abstract (English)
Multilingual large language models (LLMs) often exhibit language confusion, a tendency to generate responses in a dominant language irrespective of the prompt's language. To address this, we propose Smoothie-Qwen, a lightweight, post-hoc method that mitigates language bias without retraining. This technique selectively adjusts token-level output probabilities to effectively suppress undesired language generation. Applied to the Qwen model, our method reduces unintended Chinese output by over 95% while preserving task accuracy on multilingual benchmarks. This work provides a practical and efficient solution for enhancing the language controllability of LLMs, making them more reliable for global applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。