让视频生成模型像人一样持续学习新内容，不丢旧知识。
VidCLearn: A Continual Learning Approach for Text-to-Video Generation
- 用师生架构+生成回放，边学新视频边保留旧知识。
- 在多个数据集上，生成视频更连贯、更贴合文本描述。
- 适合需要长期更新的视频生成系统开发者使用。
文本到视频生成是生成式AI中的新兴领域，可从文本提示生成逼真且语义准确的视频。尽管当前模型在视觉质量和文本对齐方面表现优异，但通常依赖静态知识，难以在不从头训练的情况下融入新数据。为此，我们提出VidCLearn，一种基于扩散模型的文本到视频生成持续学习框架。该框架采用学生-教师架构，学生模型通过新增文本-视频对逐步更新，教师模型则通过生成回放机制保留先前学习的知识。此外，我们引入一种新型时序一致性损失以提升运动流畅性，并设计视频检索模块，在推理阶段提供结构引导。该架构还具备更高的计算效率，同时保持良好的生成性能。实验结果表明，VidCLearn在视觉质量、语义对齐和时序连贯性方面均优于基线方法。
原文摘要 · Abstract (English)
Text-to-video generation is an emerging field in generative AI, enabling the creation of realistic, semantically accurate videos from text prompts. While current models achieve impressive visual quality and alignment with input text, they typically rely on static knowledge, making it difficult to incorporate new data without retraining from scratch. To address this limitation, we propose VidCLearn, a continual learning framework for diffusion-based text-to-video generation. VidCLearn features a student-teacher architecture where the student model is incrementally updated with new text-video pairs, and the teacher model helps preserve previously learned knowledge through generative replay. Additionally, we introduce a novel temporal consistency loss to enhance motion smoothness and a video retrieval module to provide structural guidance at inference. Our architecture is also designed to be more computationally efficient than existing models while retaining satisfactory generation performance. Experimental results show VidCLearn's superiority over baseline methods in terms of visual quality, semantic alignment, and temporal coherence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。