arXiv:2509.25187cs.CV2025-09被引 3

提出新方法防止图像生成视频时信息泄露,提升跨域泛化能力

FlashI2V: Fourier-Guided Latent Shifting Prevents Conditional Image Leakage in Image-to-Video Generation

  • 通过隐空间移位让条件图像信息隐式融入,避免直接拼接
  • 利用傅里叶高频特征加速收敛并控制生成细节层次
  • 仅1.3B参数即超越多个大模型,在跨域测试中表现最优

在图像到视频生成任务中,现有方法将输入图像的完整信息与噪声隐空间拼接,导致去噪器倾向于直接依赖条件图像,引发条件信息泄露问题,造成运动迟缓和颜色不一致等性能下降。本文明确指出,该问题会导致对域内数据过拟合,降低域外场景的泛化能力。为此,提出FlashI2V方法:首先采用隐空间移位,通过从噪声隐空间减去条件图像信息,使条件以隐式方式融入;其次引入傅里叶引导机制,利用傅里叶变换获取的高频幅度特征加速收敛,并灵活调节生成视频的细节层级。实验表明,FlashI2V有效抑制了条件信息泄露,在多种I2V范式中展现出最佳泛化性能。仅含1.3B参数,其在Vbench-I2V上的动态度得分为53.01,优于CogVideoX1.5-5B-I2V与Wan2.1-I2V-14B-480P。

原文摘要 · Abstract (English)

In Image-to-Video (I2V) generation, a video is created using an input image as the first-frame condition. Existing I2V methods concatenate the full information of the conditional image with noisy latents to achieve high fidelity. However, the denoisers in these methods tend to shortcut the conditional image, which is known as conditional image leakage, leading to performance degradation issues such as slow motion and color inconsistency. In this work, we further clarify that conditional image leakage leads to overfitting to in-domain data and decreases the performance in out-of-domain scenarios. Moreover, we introduce Fourier-Guided Latent Shifting I2V, named FlashI2V, to prevent conditional image leakage. Concretely, FlashI2V consists of: (1) Latent Shifting. We modify the source and target distributions of flow matching by subtracting the conditional image information from the noisy latents, thereby incorporating the condition implicitly. (2) Fourier Guidance. We use high-frequency magnitude features obtained by the Fourier Transform to accelerate convergence and enable the adjustment of detail levels in the generated video. Experimental results show that our method effectively overcomes conditional image leakage and achieves the best generalization and performance on out-of-domain data among various I2V paradigms. With only 1.3B parameters, FlashI2V achieves a dynamic degree score of 53.01 on Vbench-I2V, surpassing CogVideoX1.5-5B-I2V and Wan2.1-I2V-14B-480P. Project page: https://pku-yuangroup.github.io/FlashI2V/

图像转视频扩散模型隐空间操作傅里叶引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。