用AI代理增强医学图像问答，让模型在数据少时也能准确诊断。
AMANDA: Agentic Medical Knowledge Augmentation for Data-Efficient Medical Visual Question Answering
- 通过分步提问分解提升图像细节理解能力
- 结合生物医学知识图谱补全专业医学知识
- 无需训练即可提升少样本场景下的诊断准确率
医学多模态大模型在医学视觉问答任务中展现巨大潜力。但在缺乏标注数据的低资源环境下，现有模型常因医学推理能力受限而表现不佳：一是忽略医学图像中的细节信息；二是无法有效融入专业医学知识。为此，我们提出AMANDA，一种无需训练的代理式医学知识增强框架，利用大语言模型代理实现医学知识增强。内在知识增强聚焦于粗粒度到细粒度的问题分解，以实现全面诊断；外在知识增强则通过生物医学知识图谱检索，使推理过程有据可依。在八个医学视觉问答基准上的大量实验表明，该方法在零样本和少样本设置下均有显著性能提升。代码已开源：https://github.com/REAL-Lab-NU/AMANDA。
原文摘要 · Abstract (English)
Medical Multimodal Large Language Models (Med-MLLMs) have shown great promise in medical visual question answering (Med-VQA). However, when deployed in low-resource settings where abundant labeled data are unavailable, existing Med-MLLMs commonly fail due to their medical reasoning capability bottlenecks: (i) the intrinsic reasoning bottleneck that ignores the details from the medical image; (ii) the extrinsic reasoning bottleneck that fails to incorporate specialized medical knowledge. To address those limitations, we propose AMANDA, a training-free agentic framework that performs medical knowledge augmentation via LLM agents. Specifically, our intrinsic medical knowledge augmentation focuses on coarse-to-fine question decomposition for comprehensive diagnosis, while extrinsic medical knowledge augmentation grounds the reasoning process via biomedical knowledge graph retrieval. Extensive experiments across eight Med-VQA benchmarks demonstrate substantial improvements in both zero-shot and few-shot Med-VQA settings. The code is available at https://github.com/REAL-Lab-NU/AMANDA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。