让医学影像反事实生成带解释，帮助医生理解病变发展预测。
Towards Interpretable Counterfactual Generation via Multimodal Autoregression
- 用自回归模型同时生成病变影像和变化说明文本。
- 在首个配对数据集上实现图像与解释的精准对齐。
- 适合临床决策支持和医学教学场景。
反事实医学图像生成可帮助临床医生探索疾病进展等假设，辅助诊疗决策。现有方法虽能生成视觉逼真的图像，但缺乏可解释性，无法验证生成结果是否真实反映假设的演变过程——这对需要可追溯推理的医疗应用至关重要。本文提出可解释反事实生成（ICG）新任务，要求联合生成符合临床假设的反事实图像及描述视觉变化的解释文本。为此，我们构建了首个包含纵向医学影像、假设进展提示与文本解释的数据集ICG-CXR。进一步提出ProgEmu模型，通过自回归机制统一生成反事实图像与解释文本。实验表明，ProgEmu在生成与进展一致的图像与解释方面表现优越，展现出显著提升临床决策支持与医学教育的潜力。
原文摘要 · Abstract (English)
Counterfactual medical image generation enables clinicians to explore clinical hypotheses, such as predicting disease progression, facilitating their decision-making. While existing methods can generate visually plausible images from disease progression prompts, they produce silent predictions that lack interpretation to verify how the generation reflects the hypothesized progression -- a critical gap for medical applications that require traceable reasoning. In this paper, we propose Interpretable Counterfactual Generation (ICG), a novel task requiring the joint generation of counterfactual images that reflect the clinical hypothesis and interpretation texts that outline the visual changes induced by the hypothesis. To enable ICG, we present ICG-CXR, the first dataset pairing longitudinal medical images with hypothetical progression prompts and textual interpretations. We further introduce ProgEmu, an autoregressive model that unifies the generation of counterfactual images and textual interpretations. We demonstrate the superiority of ProgEmu in generating progression-aligned counterfactuals and interpretations, showing significant potential in enhancing clinical decision support and medical education. Project page: https://progemu.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。