arXiv:2502.21314cs.CV2025-02被引 3

用分阶段筛选视频数据+渐进式训练,提升文本生成视频质量与效率。

Raccoon: Multi-stage Diffusion Training with Coarse-to-Fine Curating Videos

  • 分粗到细筛选视频,结合多维度评估与视觉语言模型优化对齐
  • 构建100万条高质量视频数据集,生成视频更连贯真实
  • 提出新架构与四阶段训练,兼顾画质与计算效率,适合视频生成研究者

文本到视频生成在扩散模型推动下取得显著进展,但受限于数据质量与计算资源。本文提出系统性方案,同时改进数据筛选与模型设计。构建了名为CFC-VIDS-1M的百万级高质量视频数据集,采用分阶段粗到细的筛选流程:先多维度评估视频质量,再利用视觉语言模型提升文本-视频对齐与语义丰富度。基于该数据集强调的视觉质量和时间连贯性,开发了RACCOON模型,一种采用解耦时空注意力机制的Transformer架构。模型通过渐进式四阶段训练策略,高效应对视频生成复杂性。大量实验表明,该方法在保持计算效率的同时,生成视觉美观且时间连贯的视频。数据集、代码与模型将公开发布。

原文摘要 · Abstract (English)

Text-to-video generation has demonstrated promising progress with the advent of diffusion models, yet existing approaches are limited by dataset quality and computational resources. To address these limitations, this paper presents a comprehensive approach that advances both data curation and model design. We introduce CFC-VIDS-1M, a high-quality video dataset constructed through a systematic coarse-to-fine curation pipeline. The pipeline first evaluates video quality across multiple dimensions, followed by a fine-grained stage that leverages vision-language models to enhance text-video alignment and semantic richness. Building upon the curated dataset's emphasis on visual quality and temporal coherence, we develop RACCOON, a transformer-based architecture with decoupled spatial-temporal attention mechanisms. The model is trained through a progressive four-stage strategy designed to efficiently handle the complexities of video generation. Extensive experiments demonstrate that our integrated approach of high-quality data curation and efficient training strategy generates visually appealing and temporally coherent videos while maintaining computational efficiency. We will release our dataset, code, and models.

视频生成扩散模型数据筛选Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。