让图像编辑保持结构不变,尤其适合医疗等高精度场景。
TPIE: Topology-Preserved Image Editing With Text Instructions
- 用速度场建模图像形变,实现可控的结构保持编辑。
- 在2D/3D图像上生成更真实且拓扑结构完整的图像。
- 适合医学影像、建筑图纸等需精确几何的领域使用。
在医疗等敏感领域,保持物体拓扑结构的正确性至关重要。然而,现有图像编辑模型多关注强度与纹理变化,常忽略图像中的几何结构。为此,本文提出首个通过文本指令实现拓扑与几何结构保持的图像编辑方法——TPIE。该方法将新生成样本视为输入模板的可变形变体,实现可控的结构保留编辑。TPIE框架包含两个核心模块:(i) 基于自编码器的配准网络,从成对训练图像中学习由速度场参数化的物体变换隐表示;(ii) 一种新颖的潜空间条件几何扩散模型(LCDG),高效捕捉所学变换特征的数据分布,并根据自定义文本指令进行条件生成。我们在多样化的2D和3D图像上验证了TPIE,与当前最先进方法对比,实验结果表明其生成图像在拓扑结构保持方面显著优于基线方法。代码将公开于Github。
原文摘要 · Abstract (English)
Preserving topological structures is important in real-world applications, particularly in sensitive domains such as healthcare and medicine, where the correctness of human anatomy is critical. However, most existing image editing models focus on manipulating intensity and texture features, often overlooking object geometry within images. To address this issue, this paper introduces a novel method, Topology-Preserved Image Editing with text instructions (TPIE), that for the first time ensures the topology and geometry remaining intact in edited images through text-guided generative diffusion models. More specifically, our method treats newly generated samples as deformable variations of a given input template, allowing for controllable and structure-preserving edits. Our proposed TPIE framework consists of two key modules: (i) an autoencoder-based registration network that learns latent representations of object transformations, parameterized by velocity fields, from pairwise training images; and (ii) a novel latent conditional geometric diffusion (LCDG) model efficiently capturing the data distribution of learned transformation features conditioned on custom-defined text instructions. We validate TPIE on a diverse set of 2D and 3D images and compare them with state-of-the-art image editing approaches. Experimental results show that our method outperforms other baselines in generating more realistic images with well-preserved topology. Our code will be made publicly available on Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。