用视觉语言模型解决CT金属伪影,仅需少量数据即可高效还原图像。
Leveraging Image Editing Foundation Models for Data-Efficient CT Metal Artifact Reduction
- 将去伪影任务转化为上下文推理,用LoRA微调通用视觉语言模型。
- 仅需16至128对训练样本,数据需求降低两个数量级,效果达当前最优。
- 通过多参考图像引导,避免模型误将伪影当真实物体,适合医疗影像重建场景。
高衰减植入物导致的金属伪影严重损害CT图像质量,遮蔽关键解剖结构,传统深度学习方法因需大量成对训练数据而受限。本文提出范式转变:通过参数高效的低秩适应(LoRA)微调通用视觉-语言扩散基础模型,将伪影去除重构为上下文推理任务。利用丰富的视觉先验,仅需16至128对训练样本即实现有效伪影抑制,数据需求降低两个数量级。关键发现:领域自适应对减少幻觉至关重要;若无此步骤,基础模型会将条纹伪影误认为错误自然物体(如华夫饼或培养皿)。为此,提出多参考条件策略,提供来自无关受试者的清晰解剖示例作为参考,使模型能基于类别特定上下文推断未受损解剖结构。在AAPM CT-MAR基准上的广泛评估表明,该方法在感知与放射学特征指标上均达到最先进水平。本工作证明,经恰当适配的基础模型可为可解释、数据高效的医学图像重建提供可扩展替代方案。代码已公开于https://github.com/ahmetemirdagi/CT-EditMAR。
原文摘要 · Abstract (English)
Metal artifacts from high-attenuation implants severely degrade CT image quality, obscuring critical anatomical structures and posing a challenge for standard deep learning methods that require extensive paired training data. We propose a paradigm shift: reframing artifact reduction as an in-context reasoning task by adapting a general-purpose vision-language diffusion foundation model via parameter-efficient Low-Rank Adaptation (LoRA). By leveraging rich visual priors, our approach achieves effective artifact suppression with only 16 to 128 paired training examples reducing data requirements by two orders of magnitude. Crucially, we demonstrate that domain adaptation is essential for hallucination mitigation; without it, foundation models interpret streak artifacts as erroneous natural objects (e.g., waffles or petri dishes). To ground the restoration, we propose a multi-reference conditioning strategy where clean anatomical exemplars from unrelated subjects are provided alongside the corrupted input, enabling the model to exploit category-specific context to infer uncorrupted anatomy. Extensive evaluation on the AAPM CT-MAR benchmark demonstrates that our method achieves state-of-the-art performance on perceptual and radiological-feature metrics . This work establishes that foundation models, when appropriately adapted, offer a scalable alternative for interpretable, data-efficient medical image reconstruction. Code is available at https://github.com/ahmetemirdagi/CT-EditMAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。