arXiv:2505.15257cs.CL2025-05NeurIPS被引 16

让大模型少依赖语言,专注推理,跨语言能力更强。

When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners

  • 在推理时移除语言特异性表征,实现语言与推理解耦。
  • 10个开源模型在11种语言上推理性能普遍提升。
  • 无需训练,计算开销小,适合快速优化多语言模型。

多语言推理仍是大语言模型的重大挑战,性能显著偏向高资源语言。受认知神经科学启发——人类推理主要独立于语言处理——我们假设大语言模型同样将语言与推理作为可分离的模块,可通过解耦提升多语言推理能力。为此,我们在推理阶段进行因果干预,剔除语言特异性表征。在涵盖11种类型差异大的语言的10个开源大模型上实验表明,该方法一致提升多语言推理表现。分层分析进一步证实,语言与推理表征在整个模型中可有效解耦,从而增强跨语言推理能力;而保留顶层语言特征对维持语言准确性至关重要。相比监督微调或强化学习等后训练方法,本方法无需训练、计算开销极低,仍能达到相当或更优效果。研究揭示了大模型多语言推理的内部机制,并提出一种轻量、可解释的跨语言泛化改进策略。

原文摘要 · Abstract (English)

Multilingual reasoning remains a significant challenge for large language models (LLMs), with performance disproportionately favoring high-resource languages. Drawing inspiration from cognitive neuroscience, which suggests that human reasoning functions largely independently of language processing, we hypothesize that LLMs similarly encode reasoning and language as separable components that can be disentangled to enhance multilingual reasoning. To evaluate this, we perform a causal intervention by ablating language-specific representations at inference time. Experiments on 10 open-weight LLMs spanning 11 typologically diverse languages show that this language-specific ablation consistently boosts multilingual reasoning performance. Layer-wise analyses further confirm that language and reasoning representations can be effectively disentangled throughout the model, yielding improved multilingual reasoning capabilities, while preserving top-layer language features remains essential for maintaining linguistic fidelity. Compared to post-training methods such as supervised fine-tuning or reinforcement learning, our training-free language-reasoning disentanglement achieves comparable or superior results with minimal computational overhead. These findings shed light on the internal mechanisms underlying multilingual reasoning in LLMs and suggest a lightweight and interpretable strategy for improving cross-lingual generalization.

多语言推理解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。