让AI自己感知缺失信息,生成更准确的3D脑部和心脏MRI。
Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesis
- 用自感知机制让模型识别数据缺失状态
- 在三个大型MRI数据集上优于现有方法
- 适合临床中缺少模态或切片的场景
多模态脑部MRI中的模态缺失和心脏MRI中的切片缺失,在临床实践中带来显著挑战。现有方法依赖外部指导提供缺失信息以指导生成模型,但真实临床环境难以获取可靠的手动标注。此外,显式掩码提供的信息不足以提升语义一致性。本文提出CoPeDiT,一种统一的3D MRI合成框架,通过共享的完整性感知机制,针对不同缺失场景采用任务特异性实例化。我们设计了预训练任务嵌入分词器CoPeVAE,使其学习具备完整性感知能力的判别性提示标记;并构建MDiT3D,一种专用于3D MRI合成的扩散变压器架构,有效利用这些提示标记增强3D空间中的语义一致性。在三个大规模MRI数据集上的综合评估表明,CoPeDiT在多种缺失模式下均持续优于现有先进方法,生成高质量且结构一致的MRI。代码已开源:https://github.com/JK-Liu7/CoPeDiT。
原文摘要 · Abstract (English)
Missing data problems, such as missing modalities in multi-modal brain MRI and missing slices in cardiac MRI, pose significant challenges in clinical practice. Existing methods rely on external guidance to supply detailed missing-state information for instructing generative models to synthesize missing MRIs. However, manual indicators are not always available or reliable in real-world scenarios due to the unpredictable nature of clinical environments. Moreover, these explicit masks are not informative enough to provide guidance for improving semantic consistency. In this work, we argue that generative models should infer and recognize missing states in a self-perceptive manner, enabling them to better capture subtle anatomical and pathological variations. Towards this goal, we propose CoPeDiT, a shared completeness-perception framework for 3D MRI synthesis, following a common conditioning strategy with task-specific instantiations for different missing-data scenarios. Specifically, we incorporate dedicated pretext tasks into our tokenizer, CoPeVAE, empowering it to learn completeness-aware discriminative prompt tokens, and design MDiT3D, a specialized diffusion transformer architecture for 3D MRI synthesis that effectively uses the completeness-aware prompt tokens as guidance to enhance semantic consistency in 3D space. Comprehensive evaluations on three large-scale MRI datasets demonstrate that CoPeDiT consistently improves upon state-of-the-art methods across diverse missing patterns, yielding high-fidelity and structurally consistent MRI synthesis. Our code is available at https://github.com/JK-Liu7/CoPeDiT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。