通过转移高资源语言的推理特征,提升低资源语言的数学推理能力。
Enhancing Low-Resource Language Reasoning via High-Resource Language Feature Transfer

- 用稀疏自编码器定位高资源语言中有效的推理特征。
- 将这些特征注入低资源语言推理过程,显著提升其表现。
- 无需翻译或微调,直接验证特征对推理的关键作用。
大型语言模型在不同语言上的表现存在显著差异,即使解决语义等价的任务也是如此。现有分析多将其归因于预训练数据、分词方式或基准覆盖度的差异。本文提出一种互补假说:高资源语言(HRL)能更可靠地激发对任务特定(如数学)推理有用的潜在计算,而低资源语言(LRL)可能因未充分激活这些计算而导致表现下降。为此,本文引入一种机制干预框架,通过残差流激活值上的稀疏自编码器,识别并提取在成功推理中丰富的高资源语言任务相关稀疏特征,同时过滤掉源语言和通用生成特征。随后构建引导方向,并在低资源语言推理过程中注入这些特征。干预结果表明:抑制这些特征会损害源语言推理,而激活它们可部分恢复目标语言推理性能,优于随机和非任务控制组。该框架将部分跨语言推理差距重新理解为机制激发失败而非能力缺失,并提供了一种无需翻译、微调或改变用户语言的因果可测试特征迁移路径。
原文摘要 · Abstract (English)
Large language models exhibit substantial performance variation across languages, even when solving semantically equivalent tasks. Existing analyses often treat this phenomenon as an observational disparity caused by differences in pretraining data, tokenization, or benchmark coverage. We study a complementary hypothesis: high-resource languages (HRLs) may more reliably elicit latent computations useful for task-specific (i.e. mathematical) reasoning, while lower-resource languages (LRLs) may under-activate those computations despite expressing the same task. To test this hypothesis, we introduce a mechanistic intervention framework for identifying and transferring task-relevant sparse latent features across languages. Using sparse autoencoders over residual-stream activations, we isolate features enriched in successful HRL task-specific reasoning while filtering out source-language and generic-generation features. We then construct steering directions from these features and inject them during LRL inference. The resulting interventions test whether the selected features are functionally involved in the observed reasoning gap: suppressing them should impair source-language reasoning, while activating them should partially recover target-language reasoning beyond random and non-task controls. Our framework reframes some cross-lingual reasoning gaps as failures of mechanism elicitation rather than capability absence, and offers a causally testable route to feature-mediated transfer without translation, fine-tuning, or changing the user-facing language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。