小模型与大模型协作，让低资源场景下的图文命名实体识别更准。
ReFineG: Synergizing Small Supervised Models and LLMs for Low-Resource Grounded Multimodal NER
- 用小模型学领域知识，再让大模型补漏，协同提升效果。
- 在低资源数据上达F1 0.6461，CCKS2025榜单第二。
- 适合标注少、需跨模态精准识别的场景。
地面化多模态命名实体识别（GMNER）通过联合检测文本提及并将其关联到视觉区域，扩展了传统命名实体识别。现有监督方法性能强，但依赖昂贵的多模态标注，且在低资源领域表现不佳。多模态大语言模型（MLLM）虽具强泛化能力，却因领域知识冲突，产生冗余或错误提及。为此，我们提出ReFineG，一种三阶段协同框架，将小型监督模型与冻结的MLLM结合用于低资源GMNER。训练阶段采用领域感知的数据合成策略，将LLM知识迁移至小模型，同时避免领域知识冲突；精炼阶段基于置信度机制，保留小模型的可信预测，将不确定项交由MLLM处理；接地阶段采用多模态上下文选择算法，通过类比推理增强视觉定位。在CCKS2025 GMNER共享任务中，ReFineG在线榜单以F1 0.6461排名第二，验证了其在有限标注下的有效性。
原文摘要 · Abstract (English)
Grounded Multimodal Named Entity Recognition (GMNER) extends traditional NER by jointly detecting textual mentions and grounding them to visual regions. While existing supervised methods achieve strong performance, they rely on costly multimodal annotations and often underperform in low-resource domains. Multimodal Large Language Models (MLLMs) show strong generalization but suffer from Domain Knowledge Conflict, producing redundant or incorrect mentions for domain-specific entities. To address these challenges, we propose ReFineG, a three-stage collaborative framework that integrates small supervised models with frozen MLLMs for low-resource GMNER. In the Training Stage, a domain-aware NER data synthesis strategy transfers LLM knowledge to small models with supervised training while avoiding domain knowledge conflicts. In the Refinement Stage, an uncertainty-based mechanism retains confident predictions from supervised models and delegates uncertain ones to the MLLM. In the Grounding Stage, a multimodal context selection algorithm enhances visual grounding through analogical reasoning. In the CCKS2025 GMNER Shared Task, ReFineG ranked second with an F1 score of 0.6461 on the online leaderboard, demonstrating its effectiveness with limited annotations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。