提出剪裁反向传播方法，让视频生成模型高效使用像素级损失。
ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation
- 用局部帧窗口截断反向传播，降低梯度计算范围。
- 训练内存从线性增长变为恒定，支持长视频微调。
- 适用于超分、修复、增强等需要像素级损失的任务。
近期视频扩散模型通过递归帧处理实现高质量生成，但该机制导致在像素域训练时激活值随视频序列累积，带来难以承受的内存开销。这一根本限制使长视频或高分辨率视频的像素级损失微调在计算上不可行。本文提出ChopGrad，一种用于视频解码的截断反向传播方案，将梯度计算限制在局部帧窗口内，同时保持全局一致性。我们提供了该近似方法的理论分析，并证明其可实现高效的像素级损失微调。ChopGrad将训练内存从随视频帧数线性增长（全反向传播）降至恒定，且在一系列含像素级损失的条件视频生成任务中表现优于现有最先进模型，包括视频超分辨率、视频修复、神经渲染场景增强及可控驾驶视频生成。
原文摘要 · Abstract (English)
Recent video diffusion models achieve high-quality generation through recurrent frame processing where each frame generation depends on previous frames. However, this recurrent mechanism means that training such models in the pixel domain incurs prohibitive memory costs, as activations accumulate across the entire video sequence. This fundamental limitation also makes fine-tuning these models with pixel-wise losses computationally intractable for long or high-resolution videos. This paper introduces ChopGrad, a truncated backpropagation scheme for video decoding, limiting gradient computation to local frame windows while maintaining global consistency. We provide a theoretical analysis of this approximation and show that it enables efficient fine-tuning with frame-wise losses. ChopGrad reduces training memory from scaling linearly with the number of video frames (full backpropagation) to constant memory, and compares favorably to existing state-of-the-art video diffusion models across a suite of conditional video generation tasks with pixel-wise losses, including video super-resolution, video inpainting, video enhancement of neural-rendered scenes, and controlled driving video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。