用大模型提升胃肠道内镜问诊准确率
Multimodal AI for Gastrointestinal Diagnostics: Tackling VQA in MEDVQA-GI 2025
- 基于Florence多模态大模型，融合图像与文本理解
- 在KASVIR数据集上达高精度，优于基线方法
- 适合医学AI研究者与临床辅助系统开发者
本文介绍我们在ImageCLEFmed MEDVQA 2025挑战赛子任务1中的方案，聚焦胃肠道内镜的视觉问答（VQA）。采用Florence——一个大规模多模态基础模型——作为核心架构，结合强大的视觉编码器与文本编码器，解析内镜图像并生成临床相关回答。为提升泛化能力，引入保留医学特征的领域特定增强策略，提高训练多样性。在KASVIR数据集上的实验表明，微调后的Florence模型在官方评估指标上表现优异。结果验证了大模型在医疗VQA中的潜力，并为可解释性、鲁棒性及临床集成提供了有力基准。代码已公开：https://github.com/TiwariLaxuu/VQA-Florence.git
原文摘要 · Abstract (English)
This paper describes our approach to Subtask 1 of the ImageCLEFmed MEDVQA 2025 Challenge, which targets visual question answering (VQA) for gastrointestinal endoscopy. We adopt the Florence model-a large-scale multimodal foundation model-as the backbone of our VQA pipeline, pairing a powerful vision encoder with a text encoder to interpret endoscopic images and produce clinically relevant answers. To improve generalization, we apply domain-specific augmentations that preserve medical features while increasing training diversity. Experiments on the KASVIR dataset show that fine-tuning Florence yields accurate responses on the official challenge metrics. Our results highlight the potential of large multimodal models in medical VQA and provide a strong baseline for future work on explainability, robustness, and clinical integration. The code is publicly available at: https://github.com/TiwariLaxuu/VQA-Florence.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。