测试大模型对约鲁巴语和马拉雅拉姆语的文化理解能力，发现中文英文表现好但本地语言差。
Evaluating Cultural Awareness of LLMs for Yoruba, Malayalam, and English
- 用霍夫斯泰德六维度量化大模型对区域文化的理解
- 英语文化相似度高，但马拉雅拉姆语和约鲁巴语文化差异被忽略
- 建议用文化丰富数据训练区域语言大模型，提升用户体验
尽管大语言模型在众多复杂任务中表现出色，但其对地区语言与文化的理解仍缺乏系统研究。本文评估了多种大模型对印度喀拉拉邦的马拉雅拉姆语和西非约鲁巴语文化认知能力。基于霍夫斯泰德六维文化模型（权力距离PDI、个人主义IDV、成就动机MAS、不确定性规避UAV、长期导向LTO、放纵IVR），量化分析模型响应中的文化特征。结果表明，大模型在英语上表现出较高文化一致性，但在马拉雅拉姆语和约鲁巴语中未能准确捕捉六项文化维度的差异。研究强调需构建大规模、富含文化信息的区域性语言数据集，以提升对话式大模型的用户体验，并增强基于大模型代理的市场调研有效性。
原文摘要 · Abstract (English)
Although LLMs have been extremely effective in a large number of complex tasks, their understanding and functionality for regional languages and cultures are not well studied. In this paper, we explore the ability of various LLMs to comprehend the cultural aspects of two regional languages: Malayalam (state of Kerala, India) and Yoruba (West Africa). Using Hofstede's six cultural dimensions: Power Distance (PDI), Individualism (IDV), Motivation towards Achievement and Success (MAS), Uncertainty Avoidance (UAV), Long Term Orientation (LTO), and Indulgence (IVR), we quantify the cultural awareness of LLM-based responses. We demonstrate that although LLMs show a high cultural similarity for English, they fail to capture the cultural nuances across these 6 metrics for Malayalam and Yoruba. We also highlight the need for large-scale regional language LLM training with culturally enriched datasets. This will have huge implications for enhancing the user experience of chat-based LLMs and also improving the validity of large-scale LLM agent-based market research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。