arXiv:2503.08037cs.GRcs.AI2025-03CVPR被引 30

用视频模型生成物体移动,保持光影与遮挡一致。

ObjectMover: Generative Object Movement with Video Prior

  • 将物体移动建模为序列到序列任务,复用视频生成知识。
  • 在复杂光照下仍能保持阴影、反射与物体身份一致。
  • 适合需要真实感图像编辑的设计师与研究人员。

看似简单,但在图像中移动物体实则极具挑战:需重调光照、依据视角调整姿态、准确填充遮挡区域,并确保阴影与反射同步,同时保持物体身份不变。本文提出ObjectMover,一种生成式物体移动模型。核心思路是将该任务视为序列到序列问题,微调视频生成模型以利用其跨帧一致性生成能力。为解决缺乏大规模真实数据的问题,我们使用现代游戏引擎构建数据生成流水线,合成高质量数据对。此外,提出多任务学习策略,结合真实视频数据训练以增强泛化能力。大量实验表明,ObjectMover在复杂真实场景中表现优异,能有效处理极端光照与物体运动效果。

原文摘要 · Abstract (English)

Simple as it seems, moving an object to another location within an image is, in fact, a challenging image-editing task that requires re-harmonizing the lighting, adjusting the pose based on perspective, accurately filling occluded regions, and ensuring coherent synchronization of shadows and reflections while maintaining the object identity. In this paper, we present ObjectMover, a generative model that can perform object movement in highly challenging scenes. Our key insight is that we model this task as a sequence-to-sequence problem and fine-tune a video generation model to leverage its knowledge of consistent object generation across video frames. We show that with this approach, our model is able to adjust to complex real-world scenarios, handling extreme lighting harmonization and object effect movement. As large-scale data for object movement are unavailable, we construct a data generation pipeline using a modern game engine to synthesize high-quality data pairs. We further propose a multi-task learning strategy that enables training on real-world video data to improve the model generalization. Through extensive experiments, we demonstrate that ObjectMover achieves outstanding results and adapts well to real-world scenarios.

图像编辑生成模型视频先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。