研究大模型如何理解跨文化价值观及其训练方法影响
Exploring Large Language Models on Cross-Cultural Values in Connection with Training Methodology
- 分析不同训练方法对模型跨文化判断能力的影响
- 模型更倾向西方文化，多语言训练可改善偏见
- 更大模型理解社会价值更好，小模型可用合成数据提升
大型语言模型（LLMs）与人类密切互动，需深入理解人类社会的文化价值观。本文探讨开源LLMs在不同国家多样文化价值观类别上的判断能力，及其与模型规模、训练语料、对齐方式等训练方法的关系。分析表明，LLMs在社会规范判断上接近人类，但在社会制度和进步观念上表现较弱。模型普遍偏向西方文化，通过多语言语料训练可改善这一偏差。增加模型规模有助于更好理解社会价值观，而较小模型可通过合成数据增强。研究揭示了模型设计与文化理解能力之间的关键联系。
原文摘要 · Abstract (English)
Large language models (LLMs) closely interact with humans, and thus need an intimate understanding of the cultural values of human society. In this paper, we explore how open-source LLMs make judgments on diverse categories of cultural values across countries, and its relation to training methodology such as model sizes, training corpus, alignment, etc. Our analysis shows that LLMs can judge socio-cultural norms similar to humans but less so on social systems and progress. In addition, LLMs tend to judge cultural values biased toward Western culture, which can be improved with training on the multilingual corpus. We also find that increasing model size helps a better understanding of social values, but smaller models can be enhanced by using synthetic data. Our analysis reveals valuable insights into the design methodology of LLMs in connection with their understanding of cultural values.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。