无需预设候选框，用文本引导实现精准图文实体定位
A Proposal-Free Query-Guided Network for Grounded Multimodal Named Entity Recognition
- 以文本查询为驱动，直接跨模态推理实体位置
- 在多个基准上超越现有模型，提升开放域识别精度
- 适合需要精准图文对齐的场景，如智能医疗、自动驾驶
视觉-语言联合命名实体识别（GMNER）旨在识别自然语言中命名实体的文本跨度与类型，并将其映射到对应图像区域。现有方法通常分两步进行：先用通用目标检测器生成候选区域，再匹配实体。但通用检测器独立于文本实体，常忽略细粒度目标，导致定位偏差。本文提出一种无候选框的查询引导网络（QGN），通过文本引导和跨模态交互统一多模态推理与解码过程。实验表明，QGN在多个主流基准上达到领先性能，显著提升开放域场景下的实体定位准确性。
原文摘要 · Abstract (English)
Grounded Multimodal Named Entity Recognition (GMNER) identifies named entities, including their spans and types, in natural language text and grounds them to the corresponding regions in associated images. Most existing approaches split this task into two steps: they first detect objects using a pre-trained general-purpose detector and then match named entities to the detected objects. However, these methods face a major limitation. Because pre-trained general-purpose object detectors operate independently of textual entities, they tend to detect common objects and frequently overlook specific fine-grained regions required by named entities. This misalignment between object detectors and entities introduces imprecision and can impair overall system performance. In this paper, we propose a proposal-free Query-Guided Network (QGN) that unifies multimodal reasoning and decoding through text guidance and cross- modal interaction. QGN enables accurate grounding and robust performance in open-domain scenarios. Extensive experiments demonstrate that QGN achieves top performance among compared GMNER models on widely used benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。