用户可操控关键点轨迹,实现精准可控的视频帧插值。
Framer: Interactive Frame Interpolation
- 通过调整选定关键点的运动轨迹,实现交互式帧间过渡。
- 支持图像形态差异大的场景,有效处理形状与风格变化问题。
- 提供自动模式,降低使用门槛,适合内容创作与动画设计。
我们提出 Framer 用于交互式帧插值,旨在根据用户创意生成两幅图像之间的平滑过渡帧。除输入起始帧和结束帧外,该方法可通过定制部分关键点的运动轨迹来控制过渡过程。这一设计具有两大优势:其一,引入人类交互缓解了从一幅图像变换到另一幅图像时的多重可能性问题,实现对局部运动的更精细控制;其二,关键点作为最基础的交互形式,建立跨帧对应关系,提升模型在挑战性场景(如起始帧与结束帧中物体形状和风格不同)下的表现。值得注意的是,系统还提供“自动驾驶”模式,通过一个模块自动估计关键点并优化轨迹,简化实际使用。大量实验表明,Framer 在图像变形、延时视频生成、卡通帧插值等多种应用中均表现出色。代码、模型及界面将公开,以促进后续研究。
原文摘要 · Abstract (English)
We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking the start and end frames as inputs, our approach supports customizing the transition process by tailoring the trajectory of some selected keypoints. Such a design enjoys two clear benefits. First, incorporating human interaction mitigates the issue arising from numerous possibilities of transforming one image to another, and in turn enables finer control of local motions. Second, as the most basic form of interaction, keypoints help establish the correspondence across frames, enhancing the model to handle challenging cases (e.g., objects on the start and end frames are of different shapes and styles). It is noteworthy that our system also offers an "autopilot" mode, where we introduce a module to estimate the keypoints and refine the trajectory automatically, to simplify the usage in practice. Extensive experimental results demonstrate the appealing performance of Framer on various applications, such as image morphing, time-lapse video generation, cartoon interpolation, etc. The code, the model, and the interface will be released to facilitate further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。