让大模型用用户语言推理会降低准确率，存在可量化权衡。
When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracy
- 强制模型用用户语言思考，提升可读性但影响答案正确性。
- 在多语言推理任务中，模型普遍回退到英文或产生碎片化表达。
- 仅用100个样本微调可部分缓解语言与准确率的矛盾。
近期的大规模推理模型（LRMs）在英语推理任务中表现优异，但其跨语言推理能力研究不足。对真实应用而言，推理过程若能以用户母语呈现，更利于监督与理解，因此该能力至关重要。我们在XReasoning基准上全面评估两类主流LRM，发现即使最先进的模型也常回退至英语或生成片段化推理，暴露出显著的多语言推理差距。通过提示工程强制模型使用用户语言，虽提升了可读性和可监督性，但导致答案准确率下降，揭示了语言选择与准确率之间的关键权衡。进一步实验表明，仅针对100个样本进行微调即可缓解此问题，但准确率仍有一定损失。结果凸显当前LRMs在多语言推理上的局限性，并指明未来方向。代码与数据已开源：https://github.com/Betswish/mCoT-XReasoning。
原文摘要 · Abstract (English)
Recent Large Reasoning Models (LRMs) with thinking traces have shown strong performance on English reasoning tasks. However, their ability to think in other languages is less studied. This capability is as important as answer accuracy for real world applications because users may find the reasoning trace useful for oversight only when it is expressed in their own language. We comprehensively evaluate two leading families of LRMs on our XReasoning benchmark and find that even the most advanced models often revert to English or produce fragmented reasoning in other languages, revealing a substantial gap in multilingual reasoning. Prompt based interventions that force models to reason in the users language improve readability and oversight but reduce answer accuracy, exposing an important trade off. We further show that targeted post training on just 100 examples mitigates this mismatch, though some accuracy loss remains. Our results highlight the limited multilingual reasoning capabilities of current LRMs and outline directions for future work. Code and data are available at https://github.com/Betswish/mCoT-XReasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。