解决漫画生成中身份与形状冲突导致失真的问题，实现快速高保真生成。
CaricHarmony: Contrastive Diffusion Paths for Identity-Preserving Caricature Synthesis
- 通过三条并行无干扰扩散路径，分离身份与形状条件信号。
- 生成速度低于16秒，形状相似度达0.8615，用户偏好得分7.81。
- 无需训练，支持任意草图格式，适合创意设计与快速原型制作。
基于草图的漫画生成面临根本性缺陷：当身份与形状条件同时输入扩散模型时，会产生破坏性干扰，导致结果要么平庸，要么失去识别性。我们发现其根源是去噪轨迹中的条件信号污染——竞争概率分布使平衡生成成为不可能。本文提出CaricHarmony，首个无需训练的方法，通过并行的无污染扩散路径显式解决该污染问题。推理阶段维护三条路径：$π^{\mathrm{i}}$（纯身份）、$π^{\mathrm{s}}$（纯形状）、$π^{\mathrm{i+s}}$（和谐输出）。新颖的能量函数作用于交叉注意力特征，提供梯度引导，使$π^{\mathrm{i+s}}$趋向最优平衡：$ε_{\mathrm{shape}}$通过布局与语义对齐保证草图保真度，$ε_{\mathrm{id}}$则采用抗极端扭曲的标记级对应匹配。相比需70秒/身份微调的DemoCaricature或仅限贝塞尔曲线的CaricatureBooth，CaricHarmony接受任意草图格式，生成时间低于16秒。实验表明其性能达到业界领先：在相近身份一致性下，形状CLIP得分为0.8615（对比0.8450），用户整体偏好得分为7.81（对比6.06）。本方法从根本上将身份-形状冲突重构为扩散模型中的条件信号污染问题，实现了前所未有的创作控制力与可识别性兼顾。
原文摘要 · Abstract (English)
Sketch-based caricature synthesis suffers from a fundamental failure mode: when identity and shape conditions are combined in diffusion models, they create destructive interference that causes inevitable collapse toward either bland portraits or unrecognizable distortions. We identify the root cause as \emph{condition signal contamination} -- competing probability distributions in the denoising trajectory that make balanced generation impossible. We present CaricHarmony, the first training-free method that explicitly resolves this contamination through parallel uncontaminated diffusion paths. During inference, we maintain three paths: $\mathcal{P}^{\mathrm{i}}$ (pure identity), $\mathcal{P}^{\mathrm{s}}$ (pure shape), and $\mathcal{P}^{\mathrm{i+s}}$ (harmonized output). Novel energy functions operating on cross-attention features provide gradient guidance that steers $\mathcal{P}^{\mathrm{i+s}}$ toward optimal balance: $\mathcal{E}_{\mathrm{shape}}$ ensures sketch fidelity through layout and semantic alignment, while $\mathcal{E}_{\mathrm{id}}$ employs token-level correspondence matching robust to extreme distortions. Unlike DemoCaricature requiring 70 seconds per-identity fine-tuning or CaricatureBooth constrained to Bezier curves, CaricHarmony accepts any sketch format and generates in under 16 seconds. Experiments demonstrate state-of-the-art performance: 0.8615 shape CLIP score (vs. 0.8450) under comparable identity consistency score, with 7.81 overall user preference score (vs. 6.06). Our method fundamentally reconceptualizes the ID-shape conflict as conditioning signal contamination for diffusion models, enabling unprecedented creative control while preserving recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。