arXiv:2604.12378cs.CL2026-04被引 3

让大模型用目标语言推理,还能保持性能不降。

ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance

  • 构建首个多语种推理数据集,支持跨语言推理训练。
  • 通过两阶段微调,模型可在目标语言中推理且性能不降。
  • 发现模型深层负责语言切换,强化学习更高效节能。

尽管大型语言模型具备多语言能力,其训练和推理仍以英语为主导。即使面对非英语问题,模型仍主要用英语推理,导致非英语场景下的使用失配。本文提出 ReasonXL,首个涵盖五种欧洲语言(英语、德语、法语、意大利语、西班牙语)的大规模并行推理语料库,每种语言超过两百万条对齐样本,包含提示、推理过程和最终输出,可直接监督语言特定推理。基于此,我们证明可通过简单两阶段流程(监督微调 + 可验证奖励的强化学习)实现模型完全在目标语言中推理,且性能与基线相当,通用知识损失小,跨语言迁移能力基本保留。进一步表征分析显示,模型深层存在决定语言身份的激活瓶颈,上层则集中了适应带来的权重与激活变化。强化学习相比监督微调以更小参数更新实现更大行为偏离,表明其代表性的重配置更高效。

原文摘要 · Abstract (English)

Despite advances in multilingual capabilities, most large language models (LLMs) remain English-centric in their training and, crucially, in their production of reasoning traces. Even when tasked with non-English problems, these models predominantly reason in English, creating a fundamental mismatch for non-English usage scenarios. We address this disparity directly with three contributions. (i) We introduce ReasonXL, the first large-scale parallel corpus of cross-domain reasoning traces spanning five European languages (English, German, French, Italian, and Spanish), with over two million aligned samples per language, each comprising prompts, reasoning traces, and final outputs, enabling direct supervision of language-specific reasoning. (ii) Using ReasonXL, we demonstrate that LLMs can be adapted to reason entirely in a desired target language, using a simple two-stage pipeline of supervised fine-tuning (SFT) followed by reinforcement learning with verifiable rewards (RLVR). The resulting models match or exceed baseline performance, with minimal loss in general knowledge and broadly preserved cross-lingual transfer. (iii) We conduct an extensive representational analysis of the adaptation and find a clear functional division across model depth: early layers contain an activation bottleneck that causally determines language identity, while upper layers concentrate the weight and activation changes driven by adaptation. We further find that RLVR achieves greater behavioral divergence from the base model with smaller parameter updates than SFT, suggesting a more efficient representational rerouting despite much smaller weight updates.

多语言推理生成微调强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。