揭示多语言模型性能差异的根源：是语言本身复杂，还是设计选择导致？
The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?
- 从分词、编码、数据暴露等设计角度分析性能差距成因
- 标准化处理后语言间差距显著缩小，说明多数问题源于模型设计
- 提出分词、采样、架构与评估的优化建议，适合构建公平模型的研究者
多语言语言模型承诺更广泛的自然语言处理应用，但当前系统在世界各语言间表现不均。本综述探讨这些差距持续存在的原因，判断其源于语言固有的复杂性，还是建模中的设计偏差。我们围绕两个核心问题组织文献：语言差异是否由表示与分配选择（如分词、编码、数据暴露、参数共享）引起，而非本质复杂性；哪些设计选择能缓解类型多样语言间的不平等。回顾了正字法、形态学、词汇多样性、句法、信息密度及类型距离等语言特征，并将其与具体建模机制关联。发现当分词、编码和数据暴露被规范化后，语言间差距明显缩小，表明许多看似困难的问题实为当前建模选择所致。据此提出分词、采样、架构与评估的设计建议，以支持更均衡的多语言语言模型。
原文摘要 · Abstract (English)
Multilingual language models (LMs) promise broader NLP access, yet current systems deliver uneven performance across the world's languages. This survey examines why these gaps persist and whether they reflect intrinsic linguistic difficulty or modeling artifacts. We organize the literature around two questions: do linguistic disparities arise from representation and allocation choices (e.g., tokenization, encoding, data exposure, parameter sharing) rather than inherent complexity; and which design choices mitigate inequities across typologically diverse languages. We review linguistic features, such as orthography, morphology, lexical diversity, syntax, information density, and typological distance, linking each to concrete modeling mechanisms. Gaps often shrink when segmentation, encoding, and data exposure are normalized, suggesting much apparent difficulty stems from current modeling choices. We synthesize these insights into design recommendations for tokenization, sampling, architectures, and evaluation to support more balanced multilingual LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。