让动作控制更真实:用户能自由操控角色和镜头,系统自动预测物理反应。
MoRight: Motion Control Done Right

- 分离物体运动与镜头视角,通过跨视图注意力实现独立控制
- 区分主动动作与被动反应,训练模型学习因果关系
- 支持正向推理(给动作猜结果)和逆向推理(给结果反推动作)
生成由用户指定动作驱动的视频——在自由选择视角下保持物理合理场景动态——需要两项能力:(1) 解耦的动作控制,允许用户分别控制物体运动和调整相机视角;(2) 动作因果性,确保用户驱动的动作能引发其他物体的合理反应,而非仅移动像素。现有方法在两方面均存在不足:将相机与物体运动混为单一跟踪信号,并将运动视为无因果的位移。我们提出MoRight,一个统一框架,通过解耦运动建模解决上述问题。物体运动在标准静态视角中指定,经由时间跨视图注意力转移至任意目标视角,实现相机与物体运动的解耦。进一步将运动分解为主动(用户驱动)与被动(结果)成分,训练模型从数据中学习运动因果性。推理时,用户可输入主动动作,系统预测后续反应(正向推理),或指定期望的被动结果,系统反推合理驱动动作(逆向推理),同时自由调整相机视角。在三个基准测试上,实验表明其在生成质量、运动可控性和交互感知方面达到当前最优水平。
原文摘要 · Abstract (English)
Generating motion-controlled videos--where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints--demands two capabilities: (1) disentangled motion control, allowing users to separately control the object motion and adjust camera viewpoint; and (2) motion causality, ensuring that user-driven actions trigger coherent reactions from other objects rather than merely displacing pixels. Existing methods fall short on both fronts: they entangle camera and object motion into a single tracking signal and treat motion as kinematic displacement without modeling causal relationships between object motion. We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling. Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint via temporal cross-view attention, enabling disentangled camera and object control. We further decompose motion into active (user-driven) and passive (consequence) components, training the model to learn motion causality from data. At inference, users can either supply active motion and MoRight predicts consequences (forward reasoning), or specify desired passive outcomes and MoRight recovers plausible driving actions (inverse reasoning), all while freely adjusting the camera viewpoint. Experiments on three benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。