arXiv:2604.13970cs.CV2026-04中稿 · MIDL 2026

让医学影像与报告精准对齐,识别细微病灶位置。

MApLe: Multi-instance Alignment of Diagnostic Reports and Large Medical Images

论文配图:MApLe: Multi-instance Alignment of Diagnostic Reports and Large Medical Images
图 1 · 摘自论文原文
  • 分离解剖区域与诊断发现,分块匹配图文信息。
  • 在多任务评测中优于现有模型,实现精准定位。
  • 适合医学AI研究者、临床辅助诊断系统开发者。

在诊断报告中,专家将复杂的影像数据转化为具有临床意义的信息,描述细微但关键的病理表现,并在解剖上下文中表达。报告结构相对一致,用词简练,常关联微小却重要的图像观察结果。标准视觉语言模型难以识别这些文本成分与图像中微小区域的对应关系。为此,我们提出MApLe:一种多任务、多实例的视觉语言对齐方法,克服上述局限。该方法分离解剖区域与诊断发现的概念,采用分块方式将局部图像信息与报告中的句子关联。模型包含一个捕捉句子中解剖与诊断概念的文本嵌入模块,一个基于解剖结构条件化的分块图像编码器,以及这些表征的多实例对齐机制。实验表明,MApLe能成功对齐不同图像区域与报告中的多个诊断发现。在多个下游任务上评估,其对齐性能优于当前最优基线模型。代码已开源:https://github.com/cirmuw/MApLe。

原文摘要 · Abstract (English)

In diagnostic reports, experts encode complex imaging data into clinically actionable information. They describe subtle pathological findings that are meaningful in their anatomical context. Reports follow relatively consistent structures, expressing diagnostic information with few words that are often associated with tiny but consequential image observations. Standard vision language models struggle to identify the associations between these informative text components and small locations in the images. Here, we propose "MApLe", a multi-task, multi-instance vision language alignment approach that overcomes these limitations. It disentangles the concepts of anatomical region and diagnostic finding, and links local image information to sentences in a patch-wise approach. Our method consists of a text embedding trained to capture anatomical and diagnostic concepts in sentences, a patch-wise image encoder conditioned on anatomical structures, and a multi-instance alignment of these representations. We demonstrate that MApLe can successfully align different image regions and multiple diagnostic findings in free-text reports. We show that our model improves the alignment performance compared to state-of-the-art baseline models when evaluated on several downstream tasks. The code is available at https://github.com/cirmuw/MApLe.

医学影像图文对齐多实例学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。