通过结构保持的分块解码,提升神经视频表示的重建质量与压缩效率。
Structure-Preserving Patch Decoding for Efficient Neural Video Representation
- 采用像素级分块保持全局空间结构,支持从整体到局部的解码策略。
- 在标准数据集上实现更高重建质量与更优压缩性能,优于现有基线方法。
- 适合需要高保真视频重建与高效压缩的应用场景,如视频编码与生成。
隐式神经表示(INRs)在建模复杂信号方面受到广泛关注,尤其体现在将时空坐标映射为对应值的应用中。处理视频时,将紧凑输入映射到完整帧或空间分块图像是一种有效方法,相比基于坐标的映射,该策略更好保留空间关系、降低计算开销并提升重建质量。然而,直接预测整帧常限制高频视觉细节的还原。此外,传统的均匀空间分块方法易引入边界不连续性,损害空间连贯性。本文提出基于结构保持分块(SPPs)的神经视频表示方法以解决上述问题。通过类似PixelUnshuffle的确定性像素级分割,将每帧视频分解为空间对齐的分块图像,保留全局空间结构的同时支持分块级解码。训练解码器重构这些结构化分块,实现从全局布局到局部细节的逐步重建,显著减少边界伪影并缓解朴素上采样的失真。在标准视频数据集上的实验表明,该方法在重建质量与压缩性能上均优于现有基于INR的基线方法。
原文摘要 · Abstract (English)
Implicit neural representations (INRs) are the subject of extensive research, particularly in their application to modeling complex signals by mapping spatial and temporal coordinates to corresponding values. When handling videos, mapping compact inputs to entire frames or spatially partitioned patch images is an effective approach. This strategy better preserves spatial relationships, reduces computational overhead, and improves reconstruction quality compared to coordinate-based mapping. However, predicting entire frames often limits the reconstruction of high-frequency visual details. Additionally, conventional patch-based approaches based on uniform spatial partitioning tend to introduce boundary discontinuities that degrade spatial coherence. We propose a neural video representation method based on Structure-Preserving Patches (SPPs) to address such limitations. Our method separates each video frame into patch images of spatially aligned frames through a deterministic pixel-based splitting similar to PixelUnshuffle. This operation preserves the global spatial structure while allowing patch-level decoding. We train the decoder to reconstruct these structured patches, enabling a global-to-local decoding strategy that captures the global layout first and refines local details. This effectively reduces boundary artifacts and mitigates distortions from naive upsampling. Experiments on standard video datasets demonstrate that our method achieves higher reconstruction quality and better compression performance than existing INR-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。