用事件理解提升复杂描述的图像检索准确率
EVENT-Retriever: Event-Aware Multimodal Image Retrieval for Realistic Captions
- 分阶段融合文章检索、事件感知重排序与图像匹配
- 在EVENTA 2025挑战赛中取得第一名，显著优于基线
- 适合需要理解隐含因果和时间背景的多模态应用
基于自由文本描述的事件感知图像检索面临重大挑战：模型需理解视觉特征外的潜在事件语义、上下文及现实知识。传统视觉-语言检索方法在处理抽象事件、隐含因果关系、时间背景或长篇复杂叙述时表现不佳。为此，我们提出一种多阶段检索框架，结合密集文章检索、事件感知语言模型重排序、高效图像集合构建，再通过标题引导的语义匹配与秩感知选择实现精准定位。系统采用Qwen3进行文章搜索，Qwen3-Reranker实现上下文对齐，Qwen2-VL完成图像精确打分，并使用互信息排名融合（RRF）整合多配置输出。该方法在EVENTA 2025 Grand Challenge Track 2私有测试集上取得最高顶1分数，验证了基于语言推理与多模态检索结合在复杂真实场景下的有效性。代码已公开于https://github.com/vdkhoi20/EVENT-Retriever。
原文摘要 · Abstract (English)
Event-based image retrieval from free-form captions presents a significant challenge: models must understand not only visual features but also latent event semantics, context, and real-world knowledge. Conventional vision-language retrieval approaches often fall short when captions describe abstract events, implicit causality, temporal context, or contain long, complex narratives. To tackle these issues, we introduce a multi-stage retrieval framework combining dense article retrieval, event-aware language model reranking, and efficient image collection, followed by caption-guided semantic matching and rank-aware selection. We leverage Qwen3 for article search, Qwen3-Reranker for contextual alignment, and Qwen2-VL for precise image scoring. To further enhance performance and robustness, we fuse outputs from multiple configurations using Reciprocal Rank Fusion (RRF). Our system achieves the top-1 score on the private test set of Track 2 in the EVENTA 2025 Grand Challenge, demonstrating the effectiveness of combining language-based reasoning and multimodal retrieval for complex, real-world image understanding. The code is available at https://github.com/vdkhoi20/EVENT-Retriever.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。