研究多语言模型中知识与指代的一致性，发现语言差异影响模型表现。
Are Knowledge and Reference in Multilingual Language Models Cross-Lingually Consistent?
- 用跨语言共指句测试多语言模型的知识一致性
- 不同语系和语言特征导致一致性水平差异，特定层存在瓶颈
- 代码切换训练和词对齐能显著提升一致性，适合多语言应用
跨语言一致性对于评估模型跨语言迁移能力、保持知识真实性及语言性能均衡至关重要。本文通过分析多个预训练与微调模型在包含跨语言共指陈述的代码混合数据上的表现，利用可解释性方法研究模型在跨语言环境中的行为。结果显示，多语言模型在知识一致性方面存在差异，受语系、语言特征、书写系统等因素影响，并在特定网络层出现一致性瓶颈。代码切换训练和跨语言词对齐目标表现最佳，表明跨语言对齐监督与代码切换策略对提升多语言性能与一致性具有重要意义。此外，实验还表明可通过激活修补在测试阶段实现一致性校准。
原文摘要 · Abstract (English)
Cross-lingual consistency should be considered to assess cross-lingual transferability, maintain the factuality of the model knowledge across languages, and preserve the parity of language model performance. We are thus interested in analyzing, evaluating, and interpreting cross-lingual consistency for factual knowledge. To facilitate our study, we examine multiple pretrained models and tuned models with code-mixed coreferential statements that convey identical knowledge across languages. Interpretability approaches are leveraged to analyze the behavior of a model in cross-lingual contexts, showing different levels of consistency in multilingual models, subject to language families, linguistic factors, scripts, and a bottleneck in cross-lingual consistency on a particular layer. Code-switching training and cross-lingual word alignment objectives show the most promising results, emphasizing the worthiness of cross-lingual alignment supervision and code-switching strategies for both multilingual performance and cross-lingual consistency enhancement. In addition, experimental results suggest promising result for calibrating consistency in the test time via activation patching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。