动态调整视频隐空间帧率，让生成更高效。
DLFR-VAE: Dynamic Latent Frame Rate VAE for Video Generation
- 根据内容复杂度动态调节隐空间帧率，高动态区域保留更多帧。
- 无需重新训练，可直接接入现有生成模型加速推理。
- 适合追求效率的视频生成研究者和应用开发者。
本文提出动态隐空间帧率变分自编码器（DLFR-VAE），一种无需训练的范式，可在隐空间实现自适应时间压缩。现有视频生成模型通常使用预训练VAE固定压缩率，但真实视频内容存在显著的时间非均匀性：高运动片段比静态场景包含更多信息。基于此观察，DLFR-VAE根据内容复杂度动态调整隐空间帧率。其核心创新包括：(1) 动态隐空间帧率调度器，将视频划分为时间块并依据信息论内容复杂度自适应确定最优帧率；(2) 无需训练的适配机制，可将预训练VAE架构转化为能处理可变帧率特征的动态VAE。该方法可作为即插即用模块，无缝集成至现有视频生成模型中，提升生成速度。
原文摘要 · Abstract (English)
In this paper, we propose the Dynamic Latent Frame Rate VAE (DLFR-VAE), a training-free paradigm that can make use of adaptive temporal compression in latent space. While existing video generative models apply fixed compression rates via pretrained VAE, we observe that real-world video content exhibits substantial temporal non-uniformity, with high-motion segments containing more information than static scenes. Based on this insight, DLFR-VAE dynamically adjusts the latent frame rate according to the content complexity. Specifically, DLFR-VAE comprises two core innovations: (1) A Dynamic Latent Frame Rate Scheduler that partitions videos into temporal chunks and adaptively determines optimal frame rates based on information-theoretic content complexity, and (2) A training-free adaptation mechanism that transforms pretrained VAE architectures into a dynamic VAE that can process features with variable frame rates. Our simple but effective DLFR-VAE can function as a plug-and-play module, seamlessly integrating with existing video generation models and accelerating the video generation process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。