用分步证据框架提升医疗多模态推理的可解释性与准确性
CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework
- 拆分视觉定位与推理模块，避免模型依赖捷径学习
- 在标准数据集上准确率提升10.9%，超越同类大模型
- 模拟临床流程，适合需要可信医疗AI的场景
大型视觉语言模型（VLMs）展现出强大的多模态医疗推理能力，但多数为端到端黑箱，偏离临床基于证据的分阶段工作流，阻碍临床责任落实。相比之下，专家级视觉定位模型可精准定位感兴趣区域（ROIs），提供明确可靠的证据，提升推理准确率与可信度。本文提出CARE，一种基于证据的代理式框架，以实现医疗多模态推理中的临床问责。不同于将定位与推理耦合于单一通用模型的做法，CARE将任务分解为协同子模块：小型VLM提出相关医学实体；专家实体指称分割模型生成像素级ROI证据；增强提示后的全图由接地VLM进行推理。所有VLM通过可验证奖励的强化学习优化，确保答案与证据对齐。此外，一个VLM协调器负责工具调用规划与证据-答案一致性审查，实现代理控制与最终验证。在标准医疗VQA基准测试中，CARE-Flow（无协调器）相较同规模（10B）最先进模型平均准确率提升10.9%；引入动态规划与答案审查后，CARE-Coord进一步超越高度预训练的SOTA模型5.2%。实验表明，模拟临床工作流、结合解耦专用模型与显式证据的代理框架，能带来更准确且可问责的医疗AI。
原文摘要 · Abstract (English)
Large visual language models (VLMs) have shown strong multi-modal medical reasoning ability, but most operate as end-to-end black boxes, diverging from clinicians' evidence-based, staged workflows and hindering clinical accountability. Complementarily, expert visual grounding models can accurately localize regions of interest (ROIs), providing explicit, reliable evidence that improves both reasoning accuracy and trust. In this paper, we introduce CARE, advancing Clinical Accountability in multi-modal medical Reasoning with an Evidence-grounded agentic framework. Unlike existing approaches that couple grounding and reasoning within a single generalist model, CARE decomposes the task into coordinated sub-modules to reduce shortcut learning and hallucination: a compact VLM proposes relevant medical entities; an expert entity-referring segmentation model produces pixel-level ROI evidence; and a grounded VLM reasons over the full image augmented by ROI hints. The VLMs are optimized with reinforcement learning with verifiable rewards to align answers with supporting evidence. Furthermore, a VLM coordinator plans tool invocation and reviews evidence-answer consistency, providing agentic control and final verification. Evaluated on standard medical VQA benchmarks, our CARE-Flow (coordinator-free) improves average accuracy by 10.9% over the same size (10B) state-of-the-art (SOTA). With dynamic planning and answer review, our CARE-Coord yields a further gain, outperforming the heavily pre-trained SOTA by 5.2%. Our experiments demonstrate that an agentic framework that emulates clinical workflows, incorporating decoupled specialized models and explicit evidence, yields more accurate and accountable medical AI. Project page: https://xypb.github.io/CARE-Project-Page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。