用母语者测试丹麦语大模型的文化理解力，发现训练数据不足导致文化回应偏差。
DaKultur: Evaluating the Cultural Awareness of Language Models for Danish with Native Speakers
- 让63位丹麦母语者测试模型在真实文化任务中的表现
- 使用本土数据训练使响应接受率提升两倍以上
- 适合关注多语言文化适配的研究者与AI伦理实践者
大型语言模型虽已广泛应用于多语言场景，但对非英语社区常缺乏文化敏感性，输出偏向英语中心或不恰当。为探究这一差距并区分语言与文化能力，我们首次针对中资源语言丹麦语开展文化评估研究，邀请63位人口统计特征多样化的母语者，向不同模型提出需文化理解的任务。分析1038次互动结果揭示：当前依赖自动翻译的数据不足以训练或衡量文化适应能力；而使用母语者数据训练可使响应接受率超过两倍提升。研究数据已公开发布为DaKultur——首个丹麦语文化意识数据集。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have seen widespread societal adoption. However, while they are able to interact with users in languages beyond English, they have been shown to lack cultural awareness, providing anglocentric or inappropriate responses for underrepresented language communities. To investigate this gap and disentangle linguistic versus cultural proficiency, we conduct the first cultural evaluation study for the mid-resource language of Danish, in which native speakers prompt different models to solve tasks requiring cultural awareness. Our analysis of the resulting 1,038 interactions from 63 demographically diverse participants highlights open challenges to cultural adaptation: Particularly, how currently employed automatically translated data are insufficient to train or measure cultural adaptation, and how training on native-speaker data can more than double response acceptance rates. We release our study data as DaKultur - the first native Danish cultural awareness dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。