arXiv:2606.07422cs.CLcs.AI2026-06

本地语言更易获取文化知识,但常被语言能力掩盖。

The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs

论文配图:The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
图 1 · 摘自论文原文
  • 用真实文化问题构建实验框架,分离语言能力与知识访问
  • 13个语区80余模型验证:本地语言在文化知识上具优势
  • 大模型或区域适配模型更能揭示本地语言的知识优势

大型语言模型广泛用于跨语言回答文化相关问题,但本地文化知识是通过英语还是本地语言更易获取仍不明确。现有评估存在两大局限:多依赖平行模板问题,难以反映文化知识自然呈现;原始准确率混淆了语言通用能力与语言特化知识访问。本文构建基于真实世界文化问题的受控框架,结合区域基准与本地来源数据,通过交叉问题类型(文化无关 vs. 文化特定)与查询语言(英语 vs. 本地语言),并采用共享的1PL项目反应理论模型估计能力,实现语言能力与知识访问的分离。在13个语区、约80个模型中发现:英语在文化无关问题上具稳定优势,表明英语能力更强;但扣除此差距后,本地语言在几乎所有语区-模型组合中展现出正向知识访问优势。该优势常被原始准确率掩盖,但在前沿、区域对齐或语言适配模型中更显著。结果表明,本地语言表现弱未必代表文化知识弱,本地文化知识可能更易通过本地语言获取,但被语言能力不足所遮蔽。

原文摘要 · Abstract (English)

Large language models are increasingly used to answer culturally grounded questions across languages, yet it remains unclear whether local cultural knowledge is better accessed through English or the local language. Existing evaluations face two key limitations: many rely on parallel template-based questions that may not reflect how cultural knowledge naturally appears, and raw accuracy conflates general language proficiency with language-conditioned knowledge access. We address these issues with a controlled framework built on real-world cultural questions collected from regional benchmarks and local sources. By crossing question type (culture-agnostic vs. culture-specific) with query language (English vs. local language), and estimating ability with a shared 1PL item response theory model, we separate proficiency from localized knowledge access. Across 13 locales and roughly 80 models, we find a consistent English advantage on culture-agnostic questions, indicating stronger English proficiency. However, after accounting for this proficiency gap, local languages show a positive knowledge-access advantage in nearly all locale-model settings. This advantage is often masked in raw accuracy but becomes more visible for frontier, regionally aligned, or language-adapted models. Our results suggest that weaker local-language performance does not necessarily imply weaker cultural knowledge; rather, local cultural knowledge may be more accessible through the local language but hidden by limited language proficiency.

文化知识语言能力LLM评估本地化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。