arXiv:2602.05932cs.CL2026-02Conference of the …被引 2

多语言大模型在价值观题上是统一的'通晓者',还是随语言变化的'多重身份者'?

Polyglots or Multitudes? Multilingual LLM Answers to Value-laden Multiple-Choice Questions

  • 构建8种欧洲语言的权威价值观问卷数据集MEVS,避免机器翻译偏差
  • 30多个大模型测试显示:大模型整体一致性高,但部分题目答案仍分裂
  • 指令微调模型在特定题目中出现语言依赖性,提示偏好训练有选择性影响

多项选择题常用于评估大语言模型的知识、推理能力甚至价值观。尽管多语言对事实回忆的影响已有研究,但本文探讨了更少被关注的问题:语言是否导致大模型在价值观类题目上的回答差异?多语言大模型是像理论上的通晓者一样跨语言一致,还是像多个单语模型一样,因语言不同而表达不同价值观?我们发布了新的多语言欧洲价值观调查(MEVS)数据集,其8种语言问题均由人工翻译对齐,不同于以往依赖机器翻译或临时提示的方法。我们在超过30个不同规模、厂商和对齐状态的多语言大模型上,通过控制提示变量(如选项顺序、符号类型、尾部字符),施加子集问题。结果显示,尽管更大的指令微调模型整体一致性更高,但响应鲁棒性在题目间差异显著——某些题目模型答案高度一致,而另一些则分歧明显。所有一致的指令微调模型均在特定题目中表现出语言相关行为,凸显偏好微调的选择性影响,需进一步研究。

原文摘要 · Abstract (English)

Multiple-Choice Questions (MCQs) are often used to assess knowledge, reasoning abilities, and even values encoded in large language models (LLMs). While the effect of multilingualism has been studied on LLM factual recall, this paper seeks to investigate the less explored question of language-induced variation in value-laden MCQ responses. Are multilingual LLMs consistent in their responses across languages, i.e. behave like theoretical polyglots, or do they answer value-laden MCQs depending on the language of the question, like a multitude of monolingual models expressing different values through a single model? We release a new corpus, the Multilingual European Value Survey (MEVS), which, unlike prior work relying on machine translation or ad hoc prompts, solely comprises human-translated survey questions aligned in 8 European languages. We administer a subset of those questions to over thirty multilingual LLMs of various sizes, manufacturers and alignment-fine-tuning status under comprehensive, controlled prompt variations including answer order, symbol type, and tail character. Our results show that while larger, instruction-tuned models display higher overall consistency, the robustness of their responses varies greatly across questions, with certain MCQs eliciting total agreement within and across models while others leave LLM answers split. Language-specific behavior seems to arise in all consistent, instruction-fine-tuned models, but only on certain questions, warranting a further study of the selective effect of preference fine-tuning.

多语言价值观评估大模型行为

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。