用关键帧控制3D舞蹈摄像机，让运镜更自然流畅。
DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis
- 分三阶段生成：检测关键帧、合成关键帧、预测过渡函数
- 在DCM数据集上显著优于已有方法，减少抖动和过度平滑
- 适合需要精准控制镜头运动的动画与舞蹈视频制作
从音乐和舞蹈中合成摄像机运动极具挑战性，因舞蹈电影拍摄既需连续运镜又需突然切换以模拟多机位。以往方法对每一帧同等处理，导致后期出现抖动和不可避免的平滑问题。为此，我们提出将动画师经验融入任务，将其建模为三阶段过程：关键帧检测、关键帧合成、过渡函数预测。基于此，设计了端到端的舞蹈摄像机合成框架 DanceCamAnimator，模仿人类动画流程，具备可变长度的关键帧可控性。在 DCM 数据集上的大量实验表明，该方法在定量和定性指标上均超越现有基线。代码将发布于 https://github.com/Carmenw1203/DanceCamAnimator-Official。
原文摘要 · Abstract (English)
Synthesizing camera movements from music and dance is highly challenging due to the contradicting requirements and complexities of dance cinematography. Unlike human movements, which are always continuous, dance camera movements involve both continuous sequences of variable lengths and sudden drastic changes to simulate the switching of multiple cameras. However, in previous works, every camera frame is equally treated and this causes jittering and unavoidable smoothing in post-processing. To solve these problems, we propose to integrate animator dance cinematography knowledge by formulating this task as a three-stage process: keyframe detection, keyframe synthesis, and tween function prediction. Following this formulation, we design a novel end-to-end dance camera synthesis framework \textbf{DanceCamAnimator}, which imitates human animation procedures and shows powerful keyframe-based controllability with variable lengths. Extensive experiments on the DCM dataset demonstrate that our method surpasses previous baselines quantitatively and qualitatively. Code will be available at \url{https://github.com/Carmenw1203/DanceCamAnimator-Official}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。