用DiT架构实现草图自动上色，大幅降低显存占用且效果更优。
SketchColour: Channel Concat Guided DiT-based Sketch-to-Colour Pipeline for 2D Animation
- 用通道拼接适配器+LoRA微调注入草图信息，不依赖双路ControlNet
- 仅用一半数据训练，在SAKUGA上各项指标超越现有方法
- 生成动画时间连贯，无明显颜色溢出或形变伪影，适合动画师提效
高质量2D动画制作过程高度依赖人工绘制与上色，工作量巨大。本文提出SketchColour，首个基于扩散Transformer（DiT）骨干的草图到上色流水线。通过将传统U-Net去噪器替换为DiT结构，并采用轻量级通道拼接适配器注入草图信息，结合LoRA微调，实现了无需复制ControlNet的条件控制，显著减少参数量与显存占用。在SAKUGA数据集上的评估表明，尽管训练数据仅为竞争模型的一半，其性能仍全面优于当前最优视频上色方法。该方法生成的动画具备良好的时序一致性，极少出现颜色溢出或物体形变等伪影。代码已开源：https://bconstantine.github.io/SketchColour。
原文摘要 · Abstract (English)
The production of high-quality 2D animation is highly labor-intensive process, as animators are currently required to draw and color a large number of frames by hand. We present SketchColour, the first sketch-to-colour pipeline for 2D animation built on a diffusion transformer (DiT) backbone. By replacing the conventional U-Net denoiser with a DiT-style architecture and injecting sketch information via lightweight channel-concatenation adapters accompanied with LoRA finetuning, our method natively integrates conditioning without the parameter and memory bloat of a duplicated ControlNet, greatly reducing parameter count and GPU memory usage. Evaluated on the SAKUGA dataset, SketchColour outperforms previous state-of-the-art video colourization methods across all metrics, despite using only half the training data of competing models. Our approach produces temporally coherent animations with minimal artifacts such as colour bleeding or object deformation. Our code is available at: https://bconstantine.github.io/SketchColour .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。