让文生图模型精准生成反事实图像，保持主体不变。
Causal-Adapter: Taming Text-to-Image Diffusion for Faithful Counterfactual Generation
- 基于因果建模，通过属性正则化实现精准控制。
- 在真实数据上实现87%的FID降低，生成更逼真图像。
- 适合需要可控图像编辑的研究者和开发者。
我们提出Causal-Adapter，一种模块化框架，用于将冻结的文本到图像扩散模型适配为反事实图像生成。该方法支持对目标属性进行因果干预，并一致地将其影响传递给因果依赖项，同时保留图像的核心身份。与以往依赖提示工程而无显式因果结构的方法不同，Causal-Adapter利用结构因果建模，结合两种属性正则化策略：(i) 提示对齐注入，将因果属性与文本嵌入对齐以实现精确语义控制；(ii) 条件化标记对比损失，解耦属性因子并减少虚假相关性。Causal-Adapter在合成和真实世界数据集上均达到领先性能，包括在Pendulum数据集上实现高达91%的MAE降低，用于精确属性控制；在ADNI数据集上实现高达87%的FID降低，用于高保真MRI生成。结果表明该方法具有鲁棒、可泛化的反事实编辑能力，实现忠实的属性修改与强身份保留。代码与模型将发布于：https://leitong02.github.io/causaladapter/。
原文摘要 · Abstract (English)
We present Causal-Adapter, a modular framework that adapts frozen text-to-image diffusion backbones for counterfactual image generation. Our method supports causal interventions on target attributes and consistently propagates their effects to causal dependents while preserving the core identity of the image. Unlike prior approaches that rely on prompt engineering without explicit causal structure, Causal-Adapter leverages structural causal modeling with two attribute-regularization strategies: (i) prompt-aligned injection, which aligns causal attributes with textual embeddings for precise semantic control, and (ii) a conditioned token contrastive loss that disentangles attribute factors and reduces spurious correlations. Causal-Adapter achieves state-of-the-art performance on both synthetic and real-world datasets, including up to a 91% reduction in MAE on Pendulum for accurate attribute control and up to an 87% reduction in FID on ADNI for high-fidelity MRI generation. These results demonstrate robust, generalizable counterfactual editing with faithful attribute modification and strong identity preservation. Code and models will be released at: https://leitong02.github.io/causaladapter/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。