研究多语言大模型数学推理参数的共用与差异，发现部分参数跨语言共享。
LLM Parameters for Math Across Languages: Shared or Separate?
- 通过机制分析定位不同语言的数学推理参数
- 中间层参数跨语言重叠最强，英语相关参数最多
- 低资源语言数学参数更少，体现语言依赖性
大型语言模型（LLMs）在跨语言数学推理能力上存在显著差异，但这些差异是源于语言特有参数，还是共享机制在不同语言中表现不同仍不明确。本文对多语言大模型中的数学推理进行跨语言机制分析，能够定位并比较支持数学推理的模型参数。结果表明，提取出的数学相关参数具有部分跨语言重叠，其中重叠程度最强集中于模型中间层。进一步观察发现，英语始终产生最多的数学相关参数，而低资源语言则表现出更少的相关参数集。这些结果表明，多语言大模型中的数学行为既非完全语言无关，也非完全语言特定，而是呈现系统性的语言依赖性部分共享特征。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit substantial cross-lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language-specific parameters or a shared mechanism that manifests differently by language. We present a cross-lingual mechanistic analysis of mathematical reasoning in LLMs, enabling us to localize and compare model parameters that support mathematical reasoning across languages. We find that the extracted math-associated parameters exhibit partial cross-lingual overlap, with the strongest overlap concentrated in intermediate model layers. We further observe that English consistently produces the largest set of math-relevant parameters, whereas lower-resource languages reveal smaller sets of relevant parameters. These results suggest that math-related behavior in multilingual LLMs is neither fully language-invariant nor fully language-specific, but instead exhibits partial cross-lingual parameter overlap with systematic language-dependent differences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。