用参考视频一键让手绘图动起来，保留原样还带真实动作。
SketchAnimator: Animate Sketch via Motion Customization of Text-to-Video Diffusion Models
- 分三阶段：学外观、学动作、用贝塞尔曲线优化运动轨迹。
- 仅需一张参考视频，就能让手绘图生成符合动作的动画。
- 适合设计初学者，快速实现创意动画效果。
绘画是人类表达创意的独特方式。将静态手绘图动画化能赋予其生命力，为设计师开辟新可能。但传统动画耗时费力，对新手不友好。本文提出SketchAnimator模型，通过输入一张手绘图和一段参考视频，实现「一拍即动」的动画生成。方法分为三个阶段：第一阶段学习手绘外观特征；第二阶段从参考视频中提取运动动态；第三阶段利用分数蒸馏采样（SDS）优化每帧贝塞尔曲线参数，使动画精准还原参考视频的动作。最终生成的动画既保留原始手绘风格，又呈现真实的动态效果。与现有方法对比，本方法在单次运动定制任务中表现更优，验证了其有效性。
原文摘要 · Abstract (English)
Sketching is a uniquely human tool for expressing ideas and creativity. The animation of sketches infuses life into these static drawings, opening a new dimension for designers. Animating sketches is a time-consuming process that demands professional skills and extensive experience, often proving daunting for amateurs. In this paper, we propose a novel sketch animation model SketchAnimator, which enables adding creative motion to a given sketch, like "a jumping car''. Namely, given an input sketch and a reference video, we divide the sketch animation into three stages: Appearance Learning, Motion Learning and Video Prior Distillation. In stages 1 and 2, we utilize LoRA to integrate sketch appearance information and motion dynamics from the reference video into the pre-trained T2V model. In the third stage, we utilize Score Distillation Sampling (SDS) to update the parameters of the Bezier curves in each sketch frame according to the acquired motion information. Consequently, our model produces a sketch video that not only retains the original appearance of the sketch but also mirrors the dynamic movements of the reference video. We compare our method with alternative approaches and demonstrate that it generates the desired sketch video under the challenge of one-shot motion customization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。