arXiv:2603.15003cs.CV2026-03被引 1

用少量样本让图像编辑模型学会插帧,无需改架构

Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning

  • 用低秩适配(LoRA)微调图像编辑模型,仅需64-256样本
  • 原模型完全无法生成连贯中间帧,适配后成功实现插帧
  • 适合资源受限场景下快速部署视频生成能力

预训练的图像编辑模型具备强大的空间推理和对象感知变换能力,源于数十亿图像-文本对的学习,但缺乏显式的时序建模。本文证明,这些空间先验可通过极少适应转化为时序合成能力——无需引入视频专用架构或运动估计模块。我们展示,原本仅用于静态指令编辑的大规模图像编辑模型(Qwen-Image-Edit),仅通过64至256个样本及低秩适配(LoRA)即可成功应用于视频帧插值(VFI)。核心贡献在于揭示:模型对“物体如何在静态场景中变化”的内在理解中蕴含潜在时序推理能力,可通过少样本微调激活。基线模型在生成连贯中间帧上完全失败,而参数高效适配后成功解锁插帧能力。本工作不与从头训练的专用VFI方法竞争,而是表明图像编辑基础模型在时序任务上具有未被发掘的潜力,为资源受限场景下的视频合成提供了数据高效的路径。这弥合了图像操作与视频理解之间的鸿沟,暗示空间与时序推理在基础模型中可能比以往认为的更为紧密关联。

原文摘要 · Abstract (English)

Pre-trained image editing models exhibit strong spatial reasoning and object-aware transformation capabilities acquired from billions of image-text pairs, yet they possess no explicit temporal modeling. This paper demonstrates that these spatial priors can be repurposed to unlock temporal synthesis capabilities through minimal adaptation - without introducing any video-specific architecture or motion estimation modules. We show that a large image editing model (Qwen-Image-Edit), originally designed solely for static instruction-based edits, can be adapted for Video Frame Interpolation (VFI) using only 64-256 training samples via Low-Rank Adaptation (LoRA). Our core contribution is revealing that the model's inherent understanding of "how objects transform" in static scenes contains latent temporal reasoning that can be activated through few-shot fine-tuning. While the baseline model completely fails at producing coherent intermediate frames, our parameter-efficient adaptation successfully unlocks its interpolation capability. Rather than competing with task-specific VFI methods trained from scratch on massive datasets, our work establishes that foundation image editing models possess untapped potential for temporal tasks, offering a data-efficient pathway for video synthesis in resource-constrained scenarios. This bridges the gap between image manipulation and video understanding, suggesting that spatial and temporal reasoning may be more intertwined in foundation models than previously recognized

视频插帧少样本学习图像编辑低秩适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。