用文本控制手绘线稿动画，保持动作连贯与结构稳定
Enhancing Sketch Animation: Text-to-Video Diffusion Models with Temporal Consistency and Rigidity Constraints
- 通过参数化笔画表示，结合文本引导的扩散模型生成动画
- 引入长度-面积正则化，使笔画在时序上平滑移动
- 采用刚性保持损失，防止线条变形或拓扑错乱，适合艺术创作
使用传统工具进行手绘线稿动画制作困难且复杂。线稿可作为视觉解释的基础，将其动画化能带来实时场景体验。本文提出一种基于描述性文本提示的线稿动画生成方法。该方法采用笔画的参数化表示，不同于以往难以准确估计运动并常破坏拓扑结构的方法，我们利用预训练的文生视频扩散模型，并结合SDS损失来引导笔画运动。为保证时序一致性，引入长度-面积（LA）正则化以精确估计控制点在帧序列中的平滑位移。此外，为保持形状不变、避免拓扑变化，应用形状保持的尽可能刚性（ARAP）损失以维持线稿刚性。实验表明，本方法在定量与定性评估中均优于现有最先进水平。
原文摘要 · Abstract (English)
Animating hand-drawn sketches using traditional tools is challenging and complex. Sketches provide a visual basis for explanations, and animating these sketches offers an experience of real-time scenarios. We propose an approach for animating a given input sketch based on a descriptive text prompt. Our method utilizes a parametric representation of the sketch's strokes. Unlike previous methods, which struggle to estimate smooth and accurate motion and often fail to preserve the sketch's topology, we leverage a pre-trained text-to-video diffusion model with SDS loss to guide the motion of the sketch's strokes. We introduce length-area (LA) regularization to ensure temporal consistency by accurately estimating the smooth displacement of control points across the frame sequence. Additionally, to preserve shape and avoid topology changes, we apply a shape-preserving As-Rigid-As-Possible (ARAP) loss to maintain sketch rigidity. Our method surpasses state-of-the-art performance in both quantitative and qualitative evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。