无需训练数据，让多物体素描自动动起来。
Multi-Object Sketch Animation by Scene Decomposition and Motion Planning
- 分步拆解场景并规划动作，用大模型驱动动画生成
- 在多个数据集上实现更流畅、更符合逻辑的多物体动画
- 适合动画设计、创意工具开发人员快速生成复杂动画
素描动画通过生成动态视频序列使静态草图“活”起来，广泛应用于GIF设计、卡通制作和日常娱乐。现有方法在单物体素描动画上表现良好，但在多物体场景中表现不佳。分析其失败原因，我们识别出两大挑战：物体感知的动作建模与复杂动作优化。为此，提出无需训练数据的MoSketch方法，基于迭代优化的得分蒸馏采样（SDS）实现多物体素描动画。采用分而治之策略，引入四个创新模块：基于大模型的场景分解、基于大模型的动作规划、多粒度动作细化和组合式SDS。大量定性和定量实验表明，该方法优于现有技术。MoSketch为多物体素描动画开辟了新路径，推动未来研究与应用发展。
原文摘要 · Abstract (English)
Sketch animation, which brings static sketches to life by generating dynamic video sequences, has found widespread applications in GIF design, cartoon production, and daily entertainment. While current methods for sketch animation perform well in single-object sketch animation, they struggle in multi-object scenarios. By analyzing their failures, we identify two major challenges of transitioning from single-object to multi-object sketch animation: object-aware motion modeling and complex motion optimization. For multi-object sketch animation, we propose MoSketch based on iterative optimization through Score Distillation Sampling (SDS) and thus animating a multi-object sketch in a training-data free manner. To tackle the two challenges in a divide-and-conquer strategy, MoSketch has four novel modules, i.e., LLM-based scene decomposition, LLM-based motion planning, multi-grained motion refinement, and compositional SDS. Extensive qualitative and quantitative experiments demonstrate the superiority of our method over existing sketch animation approaches. MoSketch takes a pioneering step towards multi-object sketch animation, opening new avenues for future research and applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。