arXiv:2608.16709cs.CVcs.AI2026-08

MIRROR让医学影像报告可追溯,确保每句话都有模型依据。

MIRROR: Multimodal Intelligent Radiology Reasoning and Observation Reporter

论文配图:MIRROR: Multimodal Intelligent Radiology Reasoning and Observation Reporter
图 1 · 摘自论文原文
  • 用分类器+定位器+生成器三段式结构,杜绝模型捏造发现
  • 在ChestMNIST上多标签分类宏AUROC达0.729,优于随机排序1.6~6.8倍
  • 报告内容可审计,适合需可信诊断依据的临床与研究场景

放射科医生面对模型输出时面临两大问题:模型只返回数值而无解释,任何将数值转为文字的系统都可能悄悄添加模型未识别的结论。MIRROR是一个研究原型,旨在分离这些问题。它由多标签分类器、基于Grad-CAM的定位器(将每个阳性发现映射到特定解剖区域)和报告生成器组成,后者仅接收标签、置信度和区域信息,不接触图像。由于语言层无法看到像素,无法声称分类器未检测到的发现。我们明确指出其价值:MIRROR报告中的发现可基于概率向量审计,而表述句为普通生成文本,我们展示了一个提及系统从未测量的心胸比的实例。一个统一注册表包含胸部X光、脑MRI和头颅CT的术语、解剖结构和表达方式,因此新增模态只需数据变更;三种模态均被路由并测试,其中一种被训练。在ChestMNIST上,该分类器达到宏AUROC 0.729,所有14个标签排名均优于随机,精度是随机排序的1.6至6.8倍。但在默认0.5阈值下,11个标签完全无阳性预测,其看似优秀的布里尔分数0.045,与忽略图像的预测器0.047几乎一致。区分能力真实存在,但决策无效。在放射学常见的类别不平衡背景下,聚合指标会夸大不作为模型的表现,应与基线性能对比报告。

原文摘要 · Abstract (English)

A radiologist reading a model's output faces two problems. The model returns a number and no reason, and any system that turns that number into readable prose can quietly add claims the model never made. MIRROR is a research prototype built to separate those failures. It chains a multi-label classifier, a Grad-CAM localizer that turns each positive finding into a named anatomical region, and a report writer that receives the labels, probabilities, and regions but never the image. Because the language layer cannot see pixels, it cannot assert a finding the classifier did not make. We are precise about what that buys: a MIRROR report's findings are auditable against the probability vector, while the sentences framing them are ordinary generated text, and we show one stating a cardiothoracic ratio the system never measured. One registry holds the taxonomy, anatomy, and phrasing for chest X-ray, brain MRI, and head CT, so adding a modality is a data change; all three are routed and tested, one is trained. On ChestMNIST that classifier reaches macro AUROC 0.729 and ranks better than chance on all 14 labels, at 1.6 to 6.8 times the precision a random ranker would get. Yet at the default 0.5 threshold it emits no positive prediction at all for 11 of them, and its excellent-looking Brier score of 0.045 sits beside the 0.047 earned by a predictor that ignores the image. The discrimination is real; the decisions are not. Under the class imbalance normal in radiology, aggregate metrics flatter models that do nothing, and should be reported against that floor.

医学影像可解释性生成报告多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。