构建跨语言干扰矩阵，揭示多语言模型中的不对称干扰规律
Interference Matrix: Quantifying Cross-Lingual Interference in Transformer Encoders
- 通过83种语言的成对训练，构建大规模干扰矩阵
- 发现干扰模式与书写系统相关，而非语言亲缘性或嵌入相似性
- 该矩阵可有效预测下游任务表现，指导多语言模型设计
本文对83种语言在仅编码器的Transformer模型中表现出的语言干扰进行了全面研究。通过在所有可能的语言对上训练和评估小型BERT类模型，构建了干扰矩阵，实现了跨语言干扰的大规模量化分析。结果表明，语言间的干扰具有非对称性，其模式与传统语言特征（如语言家族）或嵌入相似性等代理指标不一致，反而更与书写系统相关。此外，我们证明该干扰矩阵能有效预测下游任务的表现，为设计高性能多语言模型提供了有力工具。
原文摘要 · Abstract (English)
In this paper, we present a comprehensive study of language interference in encoder-only Transformer models across 83 languages. We construct an interference matrix by training and evaluating small BERT-like models on all possible language pairs, providing a large-scale quantification of cross-lingual interference. Our analysis reveals that interference between languages is asymmetrical and that its patterns do not align with traditional linguistic characteristics, such as language family, nor with proxies like embedding similarity, but instead better relate to script. Finally, we demonstrate that the interference matrix effectively predicts performance on downstream tasks, serving as a tool to better design multilingual models to obtain optimal performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。