arXiv:2502.19937cs.CVcs.MM2025-02CVPR被引 8

用参考图实现精准无伪影的草图上色,适合动画制作流程。

Image Referenced Sketch Colorization Based on Animation Creation Workflow

  • 基于扩散模型,分前景与背景提取颜色并独立控制。
  • 在几何不匹配参考图下仍生成高质量、无伪影结果。
  • 支持多种推理模式,适配不同动画创作场景。

草图上色在动画与数字插画制作中至关重要。现有方法存在文本引导难以准确提供色彩与风格参考、提示引导需手动操作、图像参考易产生伪影等问题。为此,我们提出一种受真实动画生产流程启发的扩散框架。该方法以草图为空间引导,以RGB图像为颜色参考,并通过空间掩码分别提取参考图的前景与背景。特别地,引入带有LoRA模块的拆分交叉注意力机制,分别对前景与背景区域训练嵌入向量的键与值,使扩散模型能独立融合前后景信息,避免干扰并消除空间伪影。推理时,通过切换激活模块设计多种可选推理模式,适配多样化使用场景。大量定性与定量实验及用户研究证实,本方法在处理几何不匹配参考图时仍能生成高质量、无伪影结果。消融实验证明各组件有效性。代码已开源:https://github.com/tellurion-kanata/colorizeDiffusion。

原文摘要 · Abstract (English)

Sketch colorization plays an important role in animation and digital illustration production tasks. However, existing methods still meet problems in that text-guided methods fail to provide accurate color and style reference, hint-guided methods still involve manual operation, and image-referenced methods are prone to cause artifacts. To address these limitations, we propose a diffusion-based framework inspired by real-world animation production workflows. Our approach leverages the sketch as the spatial guidance and an RGB image as the color reference, and separately extracts foreground and background from the reference image with spatial masks. Particularly, we introduce a split cross-attention mechanism with LoRA (Low-Rank Adaptation) modules. They are trained separately with foreground and background regions to control the corresponding embeddings for keys and values in cross-attention. This design allows the diffusion model to integrate information from foreground and background independently, preventing interference and eliminating the spatial artifacts. During inference, we design switchable inference modes for diverse use scenarios by changing modules activated in the framework. Extensive qualitative and quantitative experiments, along with user studies, demonstrate our advantages over existing methods in generating high-qualigy artifact-free results with geometric mismatched references. Ablation studies further confirm the effectiveness of each component. Codes are available at https://github.com/ tellurion-kanata/colorizeDiffusion.

草图上色扩散模型动画生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。