arXiv:2505.20287cs.CVcs.MM2025-05CVPR被引 37

让图像生成视频时运动更精准,区分物体和镜头移动。

MotionPro: A Precise Motion Controller for Image-to-Video Generation

  • 用局部区域轨迹和运动掩码实现精细运动控制
  • 在WebVid-10M和MC-Bench上显著提升生成质量
  • 构建1.1K对标注数据集,支持细粒度评估

图像到视频生成中的交互式运动控制日益流行。现有方法通常依赖大高斯核扩展运动轨迹作为条件,未明确定义运动区域,导致控制粗糙且无法分离物体与相机运动。为此,我们提出MotionPro,一种精确的运动控制器,创新性地利用区域级轨迹和运动掩码分别调控细粒度运动合成与识别运动类别(物体或相机移动)。技术上,MotionPro通过追踪模型估计训练视频的光流图,并采样区域级轨迹以模拟推理场景。不同于使用大高斯核扩展光流,其区域轨迹方法直接利用局部区域内的轨迹,实现更精确的运动建模;同时从预测光流中推导运动掩码,捕捉运动区域的整体动态。为实现自然运动控制,MotionPro通过特征调制融合区域轨迹与运动掩码,增强视频去噪效果。尤为关键的是,我们精心构建了基准数据集MC-Bench,包含1.1K用户标注的图像-轨迹对,用于评估细粒度与物体级运动控制。在WebVid-10M和MC-Bench上的大量实验验证了MotionPro的有效性。

原文摘要 · Abstract (English)

Animating images with interactive motion control has garnered popularity for image-to-video (I2V) generation. Modern approaches typically rely on large Gaussian kernels to extend motion trajectories as condition without explicitly defining movement region, leading to coarse motion control and failing to disentangle object and camera moving. To alleviate these, we present MotionPro, a precise motion controller that novelly leverages region-wise trajectory and motion mask to regulate fine-grained motion synthesis and identify target motion category (i.e., object or camera moving), respectively. Technically, MotionPro first estimates the flow maps on each training video via a tracking model, and then samples the region-wise trajectories to simulate inference scenario. Instead of extending flow through large Gaussian kernels, our region-wise trajectory approach enables more precise control by directly utilizing trajectories within local regions, thereby effectively characterizing fine-grained movements. A motion mask is simultaneously derived from the predicted flow maps to capture the holistic motion dynamics of the movement regions. To pursue natural motion control, MotionPro further strengthens video denoising by incorporating both region-wise trajectories and motion mask through feature modulation. More remarkably, we meticulously construct a benchmark, i.e., MC-Bench, with 1.1K user-annotated image-trajectory pairs, for the evaluation of both fine-grained and object-level I2V motion control. Extensive experiments conducted on WebVid-10M and MC-Bench demonstrate the effectiveness of MotionPro. Please refer to our project page for more results: https://zhw-zhang.github.io/MotionPro-page/.

图像生成视频运动控制精细生成数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。