arXiv:2601.08645cs.CL2026-01被引 3

构建跨语言多模态习语理解基准,支持跨语言迁移与图文理解对比。

A Parallel Cross-Lingual Benchmark for Multimodal Idiomaticity Understanding

  • 构建包含34种语言、超万条习语的平行多模态数据集
  • 每条习语配5张图像,覆盖从隐喻到字面意义的谱系
  • 可用于评估模型在不同语言和模态间的习语理解能力

潜在习语表达(PIEs)的意义与特定语言社群的日常经验紧密相关,是检验自然语言处理系统语言与文化理解能力的重要挑战。本文提出XMPIE,一个包含34种语言、超过一万条条目的平行多语言多模态数据集,支持对不同语言中习语表现形式与偏好模式的比较分析,以揭示共有的文化特征。该数据集允许评估同一习语在不同语言中的模型表现,并检验一种语言中的习语理解是否可迁移到另一种语言。此外,数据集支持文本与视觉模态间习语理解的交叉研究,用于衡量一种模态的理解能否推断或影响另一种模态的理解(文本与图像)。数据由语言专家创建,遵循多语言规范,每条习语均配有5张图像,涵盖从习语性到字面性的连续谱系,包括语义相关及随机干扰项。最终形成高质量的多语言多模态习语理解评测基准。

原文摘要 · Abstract (English)

Potentially idiomatic expressions (PIEs) construe meanings inherently tied to the everyday experience of a given language community. As such, they constitute an interesting challenge for assessing the linguistic (and to some extent cultural) capabilities of NLP systems. In this paper, we present XMPIE, a parallel multilingual and multimodal dataset of potentially idiomatic expressions. The dataset, containing 34 languages and over ten thousand items, allows comparative analyses of idiomatic patterns among language-specific realisations and preferences in order to gather insights about shared cultural aspects. This parallel dataset allows to evaluate model performance for a given PIE in different languages and whether idiomatic understanding in one language can be transferred to another. Moreover, the dataset supports the study of PIEs across textual and visual modalities, to measure to what extent PIE understanding in one modality transfers or implies in understanding in another modality (text vs. image). The data was created by language experts, with both textual and visual components crafted under multilingual guidelines, and each PIE is accompanied by five images representing a spectrum from idiomatic to literal meanings, including semantically related and random distractors. The result is a high-quality benchmark for evaluating multilingual and multimodal idiomatic language understanding.

习语理解多语言多模态评测基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。