研究大模型跨语言文化知识迁移，发现知识流动不均衡。
Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon
- 构建可解释框架，控制训练数据以研究跨语言文化迁移
- 低资源语言文化知识主要单向传给英语，反向流动有限
- 提出频率决定论：预训练中出现频次高的知识更易迁移
尽管已有大量研究评估大语言模型（LLMs）处理全球文化多样性能力，但其在多语言环境下获取文化知识的机制仍不清晰。本文通过研究大模型语言适配过程中文化知识的跨语言迁移，提出一种可解释的分析框架，确保训练数据透明并控制迁移效应。针对四种非英语文化进行研究，发现英语与其他高资源语言间存在双向文化知识迁移，而低资源语言仅单向将知识传递至英语，反向流动极为有限。为解释这一不对称现象，我们提出基于频率的假设：在预训练数据中出现频率更高的文化知识更容易迁移，并通过训练语料的实证分析予以验证。
原文摘要 · Abstract (English)
Despite substantial research efforts evaluating how well large language models~(LLMs) handle global cultural diversity, the mechanisms behind their cultural knowledge acquisition, particularly in multilingual settings, remain unclear. We study this question by investigating how cultural knowledge transfers across languages during language adaptation of LLMs. We introduce an interpretable framework for studying this transfer, ensuring training data transparency and controlling transfer effects. Through a study of four non-Anglophonic cultures, we observe bidirectional cultural transfer between English and other high-resource languages, while low-resource languages primarily transfer knowledge to English with limited reverse flow. To explain this asymmetric phenomenon, we propose a frequency-based hypothesis: cultural knowledge appearing more frequently in the pretraining data transfers more easily, which is supported by empirical analysis of the training corpora.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。