用超网络加速视频分层分解，新视频只需几秒即可完成训练。
HyperNVD: Accelerating Neural Video Decomposition via Hypernetworks
- 通过超网络根据视频编码生成参数，快速构建分解模型。
- 在未见视频上收敛速度提升10倍以上，仅需数秒完成训练。
- 适合需要快速编辑的影视创作与动画制作人员使用。
将视频分解为分层表示对创意产业中的视频编辑至关重要，支持对特定图层独立操作。现有方法依赖为每段视频单独训练的隐式神经表示（INRs），导致新视频处理耗时长。为此，我们提出一种元学习策略，训练通用视频分解模型以加速新视频的训练过程。模型采用超网络架构，根据视频编码嵌入生成紧凑的INR-based分解模型参数。该方法缓解了单视频过拟合问题，并显著缩短了新视频的收敛时间。代码已公开：https://hypernvd.github.io/
原文摘要 · Abstract (English)
Decomposing a video into a layer-based representation is crucial for easy video editing for the creative industries, as it enables independent editing of specific layers. Existing video-layer decomposition models rely on implicit neural representations (INRs) trained independently for each video, making the process time-consuming when applied to new videos. Noticing this limitation, we propose a meta-learning strategy to learn a generic video decomposition model to speed up the training on new videos. Our model is based on a hypernetwork architecture which, given a video-encoder embedding, generates the parameters for a compact INR-based neural video decomposition model. Our strategy mitigates the problem of single-video overfitting and, importantly, shortens the convergence of video decomposition on new, unseen videos. Our code is available at: https://hypernvd.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。