用检索增强方法让文本生成CT图像更符合解剖结构。
Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation
- 从报告中检索相似病例,用其解剖标注作为生成指导
- 相比纯文本生成,图像质量和临床一致性显著提升
- 适合需要精准解剖控制的医学影像生成场景
文本条件的体数据医学成像生成模型具有语义控制能力,但缺乏明确的解剖引导,常导致空间模糊或解剖不一致。相反,结构驱动方法虽能保证强解剖一致性,但通常依赖真实标注,而合成目标图像时这些标注不可用。我们提出一种用于文本到CT生成的检索增强方法,在现实推理条件下融合语义与解剖信息。给定放射科报告,该方法通过3D视觉-语言编码器检索语义相关的临床病例,并利用其关联的解剖标注作为结构代理。该代理通过ControlNet分支注入文本条件的潜在扩散模型,提供粗粒度解剖引导,同时保持语义灵活性。在CT-RATE数据集上的实验表明,相比仅使用文本的基线,检索增强生成提高了图像保真度和临床一致性,并额外实现了显式的空间可控性,这是此类方法原本不具备的能力。进一步分析表明,检索质量至关重要,语义对齐的代理在所有评估维度上均带来稳定提升。本工作引入了一种原则性且可扩展的机制,弥合了体数据医学图像合成中语义条件与解剖合理性之间的差距。代码已开源:https://github.com/arco-group/RAGText2CT。
原文摘要 · Abstract (English)
Text-conditioned generative models for volumetric medical imaging provide semantic control but lack explicit anatomical guidance, often resulting in outputs that are spatially ambiguous or anatomically inconsistent. In contrast, structure-driven methods ensure strong anatomical consistency but typically assume access to ground-truth annotations, which are unavailable when the target image is to be synthesized. We propose a retrieval-augmented approach for Text-to-CT generation that integrates semantic and anatomical information under a realistic inference setting. Given a radiology report, our method retrieves a semantically related clinical case using a 3D vision-language encoder and leverages its associated anatomical annotation as a structural proxy. This proxy is injected into a text-conditioned latent diffusion model via a ControlNet branch, providing coarse anatomical guidance while maintaining semantic flexibility. Experiments on the CT-RATE dataset show that retrieval-augmented generation improves image fidelity and clinical consistency compared to text-only baselines, while additionally enabling explicit spatial controllability, a capability inherently absent in such approaches. Further analysis highlights the importance of retrieval quality, with semantically aligned proxies yielding consistent gains across all evaluation axes. This work introduces a principled and scalable mechanism to bridge semantic conditioning and anatomical plausibility in volumetric medical image synthesis. Code is available at https://github.com/arco-group/RAGText2CT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。