通过在线结构重参数化提升视频表示能力,兼顾性能与效率。
Boosting Neural Video Representation via Online Structural Reparameterization
- 引入通用重参数化模块ERB,多分支并行增强模型容量。
- 训练时动态融合参数,推理阶段转为单分支结构,降低解码开销。
- 在主流数据集上平均提升0.37-2.7 dB PSNR,适合高效视频压缩场景。
神经视频表示(NVR)是提升视频存储与传输效率的有前景范式。尽管近期研究通过架构优化提升了表示能力,但多数方法设计复杂,计算开销大,难以融入其他框架,且模型容量限制导致表达力不足,形成性能瓶颈。为此,本文提出基于在线结构重参数化的NVR框架Online-RepNeRV。核心是提出通用重参数化块ERB,通过多并行卷积路径增强模型容量;采用在线重参数化策略,在训练中动态融合参数,训练后将多分支结构等效转换为单分支结构。因此,额外计算与参数复杂度仅限编码阶段,不影响解码效率。在主流视频数据集上的实验表明,该方法相比基线平均提升0.37-2.7 dB PSNR,同时保持相近的训练时间与解码速度。
原文摘要 · Abstract (English)
Neural Video Representation~(NVR) is a promising paradigm for video compression, showing great potential in improving video storage and transmission efficiency. While recent advances have made efforts in architectural refinements to improve representational capability, these methods typically involve complex designs, which may incur increased computational overhead and lack the flexibility to integrate into other frameworks. Moreover, the inherent limitation in model capacity restricts the expressiveness of NVR networks, resulting in a performance bottleneck. To overcome these limitations, we propose Online-RepNeRV, a NVR framework based on online structural reparameterization. Specifically, we propose a universal reparameterization block named ERB, which incorporates multiple parallel convolutional paths to enhance the model capacity. To mitigate the overhead, an online reparameterization strategy is adopted to dynamically fuse the parameters during training, and the multi-branch structure is equivalently converted into a single-branch structure after training. As a result, the additional computational and parameter complexity is confined to the encoding stage, without affecting the decoding efficiency. Extensive experiments on mainstream video datasets demonstrate that our method achieves an average PSNR gain of 0.37-2.7 dB over baseline methods, while maintaining comparable training time and decoding speed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。