一个模型搞定多模态图像重建与生成，统一处理各种输入输出组合。
A Unified Framework for Multimodal Image Reconstruction and Synthesis using Denoising Diffusion Models
- 将多种任务统一为虚拟补全问题，用单个无条件扩散模型实现
- 在脑部PET/MR/CT数据上表现优异，重建质量与感知效果俱佳
- 适合需要统一处理多模态医学影像的科研与临床应用
图像重建与图像合成对处理不完整多模态成像数据至关重要，但现有方法需针对不同任务使用专用模型，增加了训练与部署的复杂性。我们提出Any2all，一种统一框架，将这些异构任务建模为单一虚拟补全问题。该框架在完整的多模态数据堆栈上训练一个单一的无条件扩散模型，并在推理时根据输入灵活适应，从任意组合的干净图像或噪声测量中“补全”目标模态。我们在一个脑部PET/MR/CT数据集上验证了Any2all，结果表明其在多模态重建与合成任务上均表现出色，不仅在失真指标上达到竞争性水平，且在感知质量上优于专用方法。
原文摘要 · Abstract (English)
Image reconstruction and image synthesis are important for handling incomplete multimodal imaging data, but existing methods require various task-specific models, complicating training and deployment workflows. We introduce Any2all, a unified framework that addresses this limitation by formulating these disparate tasks as a single virtual inpainting problem. We train a single, unconditional diffusion model on the complete multimodal data stack. This model is then adapted at inference time to ``inpaint'' all target modalities from any combination of inputs of available clean images or noisy measurements. We validated Any2all on a PET/MR/CT brain dataset. Our results show that Any2all can achieve excellent performance on both multimodal reconstruction and synthesis tasks, consistently yielding images with competitive distortion-based performance and superior perceptual quality over specialized methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。