通过层替换提升多语言模型的本地推理能力，缩小与英文中心推理的差距。
Rethinking the Multilingual Reasoning Gap with Layer Swap
- 在六种语言上构建大规模多语言推理数据集，对比本地与英文中转推理效果。
- 本地推理性能差距缩小至1.9%~3.5%，远小于以往研究结果。
- 提出层交换策略，用英文模型强推理层提升本地模型表现，保持目标语言思维链。
近期的推理型大语言模型在提示为非英语时，仍主要以英语生成思维链（CoT）。已有研究表明，强制思维链使用输入语言（本地推理）会显著降低性能，而允许模型先用英语推理再用输入语言作答（英文中转推理）效果更好。然而，多数相关研究依赖推理时干预或有限的本地训练数据。本文在更大规模、同等监督条件下重新评估这一差距：构建涵盖六种语言（英语、法语、德语、西班牙语、中文、斯瓦希里语）的长序列多语言推理数据集；基于Qwen/Qwen3-8B-Base对本地与英文中转两种范式进行微调，并在数学、科学、常识与代码任务上评估。结果显示，五种非英语语言的平均本地推理差距缩小至1.9%~3.5%，显著低于以往报告。对本地专用模型的权重空间分析表明，中间层微调更新具有一致性，外层则出现分化，说明存在一个语言无关的推理核心，外层为语言特定模块。据此提出“层交换”方法：将英文专用模型中更强的推理中间层迁移到各本地模型中，从而在五种非英语语言上基本消除本地推理差距，同时保持思维链在目标语言中。所有模型与数据集均已开源。
原文摘要 · Abstract (English)
Recent reasoning Large Language Models produce a chain-of-thought (CoT) predominantly in English, even when prompted in non-English languages. Prior work suggests that forcing the CoT to remain in the input language (native reasoning) substantially degrades performance relative to allowing the model to reason in English before answering in the input language (English-pivoted reasoning). However, most studies of this native reasoning gap rely on inference-time interventions or limited native-language training data. We revisit this comparison at a larger scale and under comparable supervision. We construct long multilingual reasoning datasets across six languages (English, French, German, Spanish, Chinese and Swahili); fine-tune specialists in both native and English-pivoted regimes on top of Qwen/Qwen3-8B-Base, and evaluate across mathematics, science, general knowledge, and code. In this setting, the average native reasoning gap shrinks to 1.9--3.5% across the five non-English languages, considerably smaller than previously reported. Weight-space analysis of the native specialists reveals aligned fine-tuning updates in the middle layers and divergence in the outer layers. This points to a largely language-agnostic reasoning core surrounded by language-specific layers. Exploiting this structure, we introduce a Layer Swap: transferring the English specialist's stronger reasoning mid-layers into each native specialist, closing most of the native reasoning gap across the five non-English languages while preserving CoT in the target language. We release all models and datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。