构建超大规模多语言事件视频检索数据集，推动跨模态信息精准定位
MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval
- 基于21.8万新闻视频与3906个事件查询，覆盖视觉、音频、文本等多源信息
- 现有顶尖模型在该任务上表现不佳，表明当前系统仍难处理复杂多模态匹配
- 适合研究多语言视频理解、跨模态检索与内容生成的学者使用
从大规模多模态数据中高效检索与合成信息已成为关键挑战。现有视频检索数据集存在范围局限，主要聚焦于描述性但模糊的查询与少量专业编辑的英文视频。为填补这一空白，我们推出MultiVENT 2.0，一个大规模、多语言的事件中心视频检索基准，包含超过218,000条新闻视频和3,906个针对具体世界事件的查询。这些查询旨在获取视频中视觉、音频、嵌入文本及文本元数据中的信息，要求系统综合利用所有模态源才能完成任务。初步结果显示，现有最先进的视觉-语言模型在此任务上表现显著不足；尽管其他方法展现出潜力，但仍不足以充分应对该问题。这些发现凸显了构建更鲁棒多模态检索系统的重要性，因为高效的视频检索是实现多模态内容理解与生成的关键步骤。
原文摘要 · Abstract (English)
Efficiently retrieving and synthesizing information from large-scale multimodal collections has become a critical challenge. However, existing video retrieval datasets suffer from scope limitations, primarily focusing on matching descriptive but vague queries with small collections of professionally edited, English-centric videos. To address this gap, we introduce $\textbf{MultiVENT 2.0}$, a large-scale, multilingual event-centric video retrieval benchmark featuring a collection of more than 218,000 news videos and 3,906 queries targeting specific world events. These queries specifically target information found in the visual content, audio, embedded text, and text metadata of the videos, requiring systems leverage all these sources to succeed at the task. Preliminary results show that state-of-the-art vision-language models struggle significantly with this task, and while alternative approaches show promise, they are still insufficient to adequately address this problem. These findings underscore the need for more robust multimodal retrieval systems, as effective video retrieval is a crucial step towards multimodal content understanding and generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。