arXiv:2508.20851cs.CV2025-08被引 4

让病理诊断模型像医生一样说人话,还能指出病灶位置。

PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosis

  • 在细胞层面融合图像与文本,生成可解释的诊断描述
  • 在两个数据集上均超越现有方法,文本质量与分割精度更高
  • 适合需要透明AI辅助诊断的临床研究与医疗AI开发

基于深度学习的自动化病理诊断显著提升了诊断效率并减少了观察者间差异,但其临床应用受限于模型决策不透明、缺乏可追溯的推理过程。为解决此问题,近期多模态视觉推理架构在像素级生成分割掩码的同时,输出语义对齐的文本解释。通过定位病变区域并生成专家风格的诊断叙述,这类模型提供了可信赖的AI辅助病理诊断所需的可解释性。在此基础上,我们提出PathMR——一种细胞级别的多模态视觉推理框架,用于病理图像分析。给定一张病理图像和一段文本查询,PathMR能同时生成专家级诊断解释并预测细胞分布模式。为评估性能,我们在公开的PathGen数据集及新构建的GADVR数据集上进行了实验。大量测试表明,PathMR在文本生成质量、分割准确率和跨模态对齐方面均持续优于当前最优的视觉推理方法,凸显其在提升AI驱动病理诊断可解释性方面的潜力。代码将开源于https://github.com/zhangye-zoe/PathMR。

原文摘要 · Abstract (English)

Deep learning based automated pathological diagnosis has markedly improved diagnostic efficiency and reduced variability between observers, yet its clinical adoption remains limited by opaque model decisions and a lack of traceable rationale. To address this, recent multimodal visual reasoning architectures provide a unified framework that generates segmentation masks at the pixel level alongside semantically aligned textual explanations. By localizing lesion regions and producing expert style diagnostic narratives, these models deliver the transparent and interpretable insights necessary for dependable AI assisted pathology. Building on these advancements, we propose PathMR, a cell-level Multimodal visual Reasoning framework for Pathological image analysis. Given a pathological image and a textual query, PathMR generates expert-level diagnostic explanations while simultaneously predicting cell distribution patterns. To benchmark its performance, we evaluated our approach on the publicly available PathGen dataset as well as on our newly developed GADVR dataset. Extensive experiments on these two datasets demonstrate that PathMR consistently outperforms state-of-the-art visual reasoning methods in text generation quality, segmentation accuracy, and cross-modal alignment. These results highlight the potential of PathMR for improving interpretability in AI-driven pathological diagnosis. The code will be publicly available in https://github.com/zhangye-zoe/PathMR.

病理诊断多模态可解释性视觉推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。