发现多语言模型答错题的根源并修复，提升低语种事实召回准确率超35%。
Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline
- 用机制分析发现模型依赖英语记忆路径处理多语言问题
- 修复路径错误后，最低性能语言的召回准确率提升超35%
- 提出无需语言数据的向量干预，通用性强且可复用
多语言大语言模型在不同语言间常出现事实不一致，尤其在非英语任务上表现明显下降。我们通过机制分析揭示其内部工作流程：模型先用英语为中心的事实召回机制处理多语言查询，再将英文答案翻译回目标语言。我们识别出两大错误来源：未能充分激活可靠的英语事实召回机制，以及从英语到目标语言的翻译错误。为此，我们提出两种与语言和数据集无关的向量干预方法，引导模型走向更优内部路径。联合使用后，最低性能语言的召回准确率提升超过35%。研究证明，机制洞察能有效释放大模型潜在的多语言能力。
原文摘要 · Abstract (English)
Multilingual large language models (LLMs) often exhibit factual inconsistencies across languages, with significantly better performance in factual recall tasks in English than in other languages. The causes of these failures, however, remain poorly understood. Using mechanistic analysis techniques, we uncover the underlying pipeline that LLMs employ, which involves using the English-centric factual recall mechanism to process multilingual queries and then translating English answers back into the target language. We identify two primary sources of error: insufficient engagement of the reliable English-centric mechanism for factual recall, and incorrect translation from English back into the target language for the final answer. To address these vulnerabilities, we introduce two vector interventions, both independent of languages and datasets, to redirect the model toward better internal paths for higher factual consistency. Our interventions combined increase the recall accuracy by over 35 percent for the lowest-performing language. Our findings demonstrate how mechanistic insights can be used to unlock latent multilingual capabilities in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。