多语言大模型关键推理其实用英语思维，用户却看不见。
Do Multilingual LLMs Think In English?
- 用对数透镜分析内部表示，发现模型先转成英语表征
- 在法德荷汉语中，核心词的表示最接近英语
- 用英语向量控制激活更有效，说明模型本质偏英语
大型语言模型具备多语言能力，能跨语言完成任务。但我们发现，当前模型在做出关键决策时，其内部表示空间最接近英语，无论输入输出语言为何。通过在法语、德语、荷兰语和中文句子上使用对数透镜分析内部表示，我们发现模型在将语义密集词汇转化为目标语言前，会先生成接近英语的表示。进一步实验表明，当控制向量在英语中计算时，激活调控效果更好。这表明多语言大模型的关键推理步骤在一种受英语主导的表示空间中进行，且这种机制对使用者不透明。
原文摘要 · Abstract (English)
Large language models (LLMs) have multilingual capabilities and can solve tasks across various languages. However, we show that current LLMs make key decisions in a representation space closest to English, regardless of their input and output languages. Exploring the internal representations with a logit lens for sentences in French, German, Dutch, and Mandarin, we show that the LLM first emits representations close to English for semantically-loaded words before translating them into the target language. We further show that activation steering in these LLMs is more effective when the steering vectors are computed in English rather than in the language of the inputs and outputs. This suggests that multilingual LLMs perform key reasoning steps in a representation that is heavily shaped by English in a way that is not transparent to system users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。