构建跨文化多模态隐喻数据集,提升模型对不同文化隐喻的理解能力。
Cultural Bias Matters: A Cross-Cultural Benchmark Dataset and Sentiment-Enriched Model for Understanding Multimodal Metaphors
- 构建包含8461对图文的中英双语多模态隐喻数据集
- 提出融合情感嵌入的SEMD模型,提升跨文化隐喻检测效果
- 揭示文化偏见对隐喻理解的影响,适合关注公平AI的研究者
隐喻在交流中普遍存在,对自然语言处理至关重要。以往自动隐喻处理研究主要依赖英语数据,常反映欧美文化偏见,导致模型性能被高估,阻碍NLP发展。然而,文化偏见对多模态隐喻处理的影响仍不明确。为此,我们提出MultiMM——一个用于中英跨文化研究的多模态隐喻数据集,包含8,461个图文广告对,每对均配有细粒度标注,深化对跨文化多模态隐喻的理解。此外,我们提出情感增强隐喻检测(SEMD)模型,通过整合情感嵌入提升跨文化背景下的隐喻理解能力。实验验证了SEMD在隐喻检测与情感分析任务中的有效性。本工作旨在提高对NLP中文化偏见的认知,推动更公平、包容的语言模型发展。数据集与代码已开源:https://github.com/DUTIR-YSQ/MultiMM。
原文摘要 · Abstract (English)
Metaphors are pervasive in communication, making them crucial for natural language processing (NLP). Previous research on automatic metaphor processing predominantly relies on training data consisting of English samples, which often reflect Western European or North American biases. This cultural skew can lead to an overestimation of model performance and contributions to NLP progress. However, the impact of cultural bias on metaphor processing, particularly in multimodal contexts, remains largely unexplored. To address this gap, we introduce MultiMM, a Multicultural Multimodal Metaphor dataset designed for cross-cultural studies of metaphor in Chinese and English. MultiMM consists of 8,461 text-image advertisement pairs, each accompanied by fine-grained annotations, providing a deeper understanding of multimodal metaphors beyond a single cultural domain. Additionally, we propose Sentiment-Enriched Metaphor Detection (SEMD), a baseline model that integrates sentiment embeddings to enhance metaphor comprehension across cultural backgrounds. Experimental results validate the effectiveness of SEMD on metaphor detection and sentiment analysis tasks. We hope this work increases awareness of cultural bias in NLP research and contributes to the development of fairer and more inclusive language models. Our dataset and code are available at https://github.com/DUTIR-YSQ/MultiMM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。