NLLB-200模型隐含了人类语言的共性概念结构，揭示多语言间深层语义关联。
Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
- 通过跨语言词表对比，发现模型嵌入距离与语言谱系距离显著相关
- 共用词汇对在模型中相似度更高，体现普遍概念关联性
- 模型保留跨语言关系结构，适合研究多语种语义统一机制
我们探究Meta的NLLB-200（200种语言）翻译模型是否学习到语言通用的概念表示。基于135种语言的Swadesh核心词汇，发现模型嵌入距离与自动化相似性判断程序（ASJP）的谱系距离显著相关（ρ=0.13，p=0.020），表明模型隐式习得了人类语言的演化结构。使用CLICS数据库中频繁共用词汇对，其嵌入相似度显著高于非共用对（U=42656，p=1.33×10⁻¹¹，d=0.96），说明模型内化了普遍概念关联。对每种语言进行均值中心化后，概念间距离比概念内距离提升1.19倍，支持语言中立概念存储的存在，类似双语脑成像中的前颞叶枢纽。基础概念对（如男/女、大/小）的语义偏移向量在跨语言间具高度一致性（平均余弦=0.84），表明二阶关系结构在类型差异大的语言中得以保留。我们发布了开源交互工具InterpretCognates及完整可复现分析流程。
原文摘要 · Abstract (English)
Do neural machine translation models learn language-universal conceptual representations, or do they merely cluster languages by surface similarity? We investigate this question by probing the representation geometry of Meta's NLLB-200, a 200-language encoder-decoder Transformer, through six experiments that bridge NLP interpretability with cognitive science theories of multilingual lexical organization. Using the Swadesh core vocabulary list embedded across 135 languages, we find that the model's embedding distances significantly correlate with phylogenetic distances from the Automated Similarity Judgment Program ($ρ= 0.13$, $p = 0.020$), demonstrating that NLLB-200 has implicitly learned the genealogical structure of human languages. We show that frequently colexified concept pairs from the CLICS database exhibit significantly higher embedding similarity than non-colexified pairs ($U = 42656$, $p = 1.33 \times 10^{-11}$, $d = 0.96$), indicating that the model has internalized universal conceptual associations. Per-language mean-centering of embeddings improves the between-concept to within-concept distance ratio by a factor of 1.19, providing geometric evidence for a language-neutral conceptual store analogous to the anterior temporal lobe hub identified in bilingual neuroimaging. Semantic offset vectors between fundamental concept pairs (e.g., man to woman, big to small) show high cross-lingual consistency (mean cosine = 0.84), suggesting that second-order relational structure is preserved across typologically diverse languages. We release InterpretCognates, an open-source interactive toolkit for exploring these phenomena, alongside a fully reproducible analysis pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。