arXiv:2509.24353cs.CV2025-09被引 1

用神经网络权重生成视频,实现高效高质量的隐式视频合成。

NeRV-Diffusion: Diffuse Implicit Neural Representations for Video Synthesis

  • 通过超网络将视频压缩为统一的神经参数,替代传统帧级特征编码。
  • 在隐式神经表示权重空间进行扩散去噪,无需跨帧注意力机制。
  • 支持平滑插值,适合需要连续帧生成的应用场景。

我们提出NeRV-Diffusion,一种基于隐式潜在视频扩散模型的方法,通过生成神经网络权重来合成视频。这些生成的权重可重组为卷积神经网络的参数,形成隐式神经表示(INR),并以帧索引作为输入解码出视频。该框架包含两个阶段:1)基于超网络的分词器,将原始视频从像素空间编码为神经参数空间,瓶颈潜变量作为INR权重进行解码;2)隐式扩散变压器,在潜变量INR权重上执行去噪。与传统逐帧编码视频的分词器不同,NeRV-Diffusion将视频整体压缩并生成为统一的神经网络,从而在去噪和解码阶段避免了时间维度的跨帧注意力,提升效率与质量。为实现高表达力且服从高斯分布的INR权重,我们复用所有NeRV层的瓶颈潜变量,并重构其权重分配、上采样连接与输入坐标。同时引入信噪比自适应损失加权和调度采样策略,有效训练隐式扩散模型。在真实世界视频基准如UCF-101和Kinetics-600上,该方法在基于INR的模型中表现最优,且与多数最新非隐式模型性能相当。此外,其具备平滑的INR权重空间,支持帧间或视频间的无缝插值。

原文摘要 · Abstract (English)

We present NeRV-Diffusion, an implicit latent video diffusion model that synthesizes videos via generating neural network weights. The generated weights can be rearranged as the parameters of a convolutional neural network, which forms an implicit neural representation (INR), and decodes into videos with frame indices as the input. Our framework consists of two stages: 1) A hypernetworkbased tokenizer that encodes raw videos from pixel space to neural parameter space, where the bottleneck latent serves as INR weights to decode. 2) An implicit diffusion transformer that denoises on the latent INR weights. In contrast to traditional video tokenizers that encode videos into frame-wise feature maps, NeRV-Diffusion compresses and generates a video holistically as a unified neural network. This enables efficient and high-quality video synthesis via obviating temporal cross-frame attentions in the denoiser and decoding video latent with dedicated decoders. To achieve Gaussian-distributed INR weights with high expressiveness, we reuse the bottleneck latent across all NeRV layers, as well as reform its weight assignment, upsampling connection and input coordinates. We also introduce SNR-adaptive loss weighting and scheduled sampling for effective training of the implicit diffusion model. NeRV-Diffusion reaches superior video generation quality over previous INR-based models and comparable performance to most recent state-of-the-art non-implicit models on real-world video benchmarks including UCF-101 and Kinetics-600. It also brings a smooth INR weight space that facilitates seamless interpolations between frames or videos.

视频生成隐式表示扩散模型权重生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。