用一张草图生成动画,自动补帧上色,大幅减少人工工作量。
ToonComposer: Streamlining Cartoon Production with Generative Post-Keyframing
- 将补帧和上色合为一步,用稀疏草图控制动作与色彩
- 仅需一张草图加一个着色参考帧即可生成高质量动画
- 适合动画师在真实项目中快速迭代,提升创作效率
传统动漫制作包含关键帧、补间和上色三个阶段,需大量手工操作。尽管近年有AI进展,但现有方法多分步处理,导致误差累积和图像瑕疵。例如,补间方法难以应对大运动,上色模型需每帧密集草图。为此,我们提出ToonComposer,一种统一补间与上色的生成式后关键帧模型。该模型采用稀疏草图注入机制,仅需关键帧草图即可精准控制。同时,通过空间低秩适配器对现代视频基础模型进行卡通化改造,保留其时间一致性先验。只需单张草图与一个着色参考帧,ToonComposer即能生成高质量动画;亦支持任意时间点多草图输入,实现更精确的动作控制。此双重能力显著降低人工负担,增强灵活性。为评估模型,我们构建了PKBench基准,包含真人手绘草图,模拟真实使用场景。结果表明,ToonComposer在视觉质量、动作连贯性与生产效率方面均优于现有方法,为AI辅助动漫制作提供更优、更灵活的解决方案。
原文摘要 · Abstract (English)
Traditional cartoon and anime production involves keyframing, inbetweening, and colorization stages, which require intensive manual effort. Despite recent advances in AI, existing methods often handle these stages separately, leading to error accumulation and artifacts. For instance, inbetweening approaches struggle with large motions, while colorization methods require dense per-frame sketches. To address this, we introduce ToonComposer, a generative model that unifies inbetweening and colorization into a single post-keyframing stage. ToonComposer employs a sparse sketch injection mechanism to provide precise control using keyframe sketches. Additionally, it uses a cartoon adaptation method with the spatial low-rank adapter to tailor a modern video foundation model to the cartoon domain while keeping its temporal prior intact. Requiring as few as a single sketch and a colored reference frame, ToonComposer excels with sparse inputs, while also supporting multiple sketches at any temporal location for more precise motion control. This dual capability reduces manual workload and improves flexibility, empowering artists in real-world scenarios. To evaluate our model, we further created PKBench, a benchmark featuring human-drawn sketches that simulate real-world use cases. Our evaluation demonstrates that ToonComposer outperforms existing methods in visual quality, motion consistency, and production efficiency, offering a superior and more flexible solution for AI-assisted cartoon production.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。