用预训练扩散模型提取语义特征，提升医学图像配准精度。
Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models
- 用扩散模型特征替代传统强度相似性损失
- 在多模态2D与单模态3D任务中均实现更优配准效果
- 特别适合结构缺失或不一致的复杂配准场景
扩散模型虽以图像生成为目标训练，却可作为下游任务的强大特征提取器。我们发现，仅用于生成自然RGB图像的现成扩散模型，能识别医学图像中的语义对应关系。基于此，我们提出利用扩散模型特征作为相似性度量，指导可变形图像配准网络。传统基于强度的相似性损失在某些解剖结构在一张图像中存在而另一张中缺失等挑战场景下常失效，导致解剖不准确的对齐。相比之下，本方法能识别真实语义对应，对齐有意义结构，忽略跨图像不存在的结构。我们在两个任务上验证了该方法的优势：多模态2D配准（DXA到X-Ray）和单模态3D配准（去脑与非去脑MRI）。代码已开源。
原文摘要 · Abstract (English)
Diffusion models, while trained for image generation, have emerged as powerful foundational feature extractors for downstream tasks. We find that off-the-shelf diffusion models, trained exclusively to generate natural RGB images, can identify semantically meaningful correspondences in medical images. Building on this observation, we propose to leverage diffusion model features as a similarity measure to guide deformable image registration networks. We show that common intensity-based similarity losses often fail in challenging scenarios, such as when certain anatomies are visible in one image but absent in another, leading to anatomically inaccurate alignments. In contrast, our method identifies true semantic correspondences, aligning meaningful structures while disregarding those not present across images. We demonstrate superior performance of our approach on two tasks: multimodal 2D registration (DXA to X-Ray) and monomodal 3D registration (brain-extracted to non-brain-extracted MRI). Code: https://github.com/uncbiag/dgir
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。