用检索增强生成提升医学影像问答准确率
Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQA
- 用通用大模型结合领域内图文示例进行检索增强生成
- 在51个提交中获第3名，平均得分41.37%
- 无需额外训练，仅靠简单索引融合即可提升临床问答质量
医学视觉问答（MedVQA）允许对医学图像进行自然语言查询，以支持临床决策和患者护理。MEDIQA-WV 2025 共享任务聚焦于伤口护理问答，要求系统从图像和患者提问中生成自由文本回答及结构化伤口属性。我们提出 MasonNLP 系统，采用通用领域指令微调的大语言模型，结合检索增强生成（RAG）框架，引入领域内文本与视觉示例。该方法使输出基于临床相关范例，显著提升推理能力、格式遵循度与响应质量，在 dBLEU、ROUGE、BERTScore 和基于 LLM 的指标上均表现优异。最佳系统在19支队伍、51个提交中位列第3，平均得分为41.37%。结果表明，仅通过轻量级 RAG（无额外训练、无复杂重排序，仅通过简单索引与融合添加少量相关示例），即可为多模态临床 NLP 任务提供高效可靠基线。
原文摘要 · Abstract (English)
Medical Visual Question Answering (MedVQA) enables natural language queries over medical images to support clinical decision-making and patient care. The MEDIQA-WV 2025 shared task addressed wound-care VQA, requiring systems to generate free-text responses and structured wound attributes from images and patient queries. We present the MasonNLP system, which employs a general-domain, instruction-tuned large language model with a retrieval-augmented generation (RAG) framework that incorporates textual and visual examples from in-domain data. This approach grounds outputs in clinically relevant exemplars, improving reasoning, schema adherence, and response quality across dBLEU, ROUGE, BERTScore, and LLM-based metrics. Our best-performing system ranked 3rd among 19 teams and 51 submissions with an average score of 41.37%, demonstrating that lightweight RAG with general-purpose LLMs -- a minimal inference-time layer that adds a few relevant exemplars via simple indexing and fusion, with no extra training or complex re-ranking -- provides a simple and effective baseline for multimodal clinical NLP tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。