提出KD-NVC框架,高效加速神经视频编码并保持压缩性能。
KD-NVC: A Search-and-Distill Framework to Accelerate Neural Video Coding

- 分模块搜索最优轻量化结构,动态分配加速资源。
- 实测在RTX 5060上实现1080p下69帧/秒解码。
- 通过能量感知损失保留率约束下的稀疏特征模式,适合边缘部署。
尽管神经视频编码(NVC)已取得优异的率失真性能,但边缘设备上的实时解码仍受限于高计算复杂度。知识蒸馏(KD)常用于模型加速,但在NVC中面临两大挑战:一是子模块异质性强,统一剪枝效果差,需按模块定制设计;二是现有蒸馏方法忽略率约束导致的特征能量稀疏性,影响压缩效率。为此,本文提出两阶段蒸馏框架KD-NVC。第一阶段引入基于加速效率的神经架构搜索(AE-NAS),探索模块级帕累托前沿,自适应分配加速预算,并通过加速效率指标在不训练所有候选架构的情况下确定最终学生模型。第二阶段设计能量感知特征蒸馏(EFD)损失,对齐师生模型的空间聚合特征能量签名,传递率约束诱导的稀疏模式,提升压缩效率。实验表明,该框架优于现有面向编码器的蒸馏方法,在RTX 5060上实现1080p下69帧/秒解码,且率失真性能与VTM-LDB相当。
原文摘要 · Abstract (English)
While neural video coding (NVC) has achieved remarkable rate-distortion performance, real-time decoding on edge devices has become an important demand but remains limited by high complexity. Knowledge distillation (KD) is widely used for model acceleration, yet its application to NVC faces critical challenges. Specifically, the heterogeneity of NVC sub-modules renders uniform architectural reduction suboptimal, necessitating a per-module design for better rate-distortion-speed trade-off. However, searching for diverse architectures via existing neural architecture search (NAS) algorithms is unaffordable due to the expensive training cost of neural video codecs. Moreover, after the lightweight architecture is determined, existing distillation methods overlook the feature-energy sparsity induced by the rate-constraint, which is essential for maintaining compression performance. To address these issues, we propose a two-stage distillation framework KD-NVC. In the first stage, we introduce an acceleration-efficiency-based neural architecture search (AE-NAS) algorithm. It explores the module-wise Pareto frontier to adaptively allocate the acceleration budget across heterogeneous modules. Also, it introduces the acceleration-efficiency metric to determine the final student architecture without practically training all architecture-level candidates. In the second stage, we design an energy-aware feature distillation (EFD) loss that aligns the spatially-aggregated feature-energy signatures between the teacher and student codecs, transferring the rate-induced sparsity patterns for better compression efficiency. Experimental results demonstrate that the proposed framework consistently outperforms existing codec-oriented distillation methods, and achieves 69 FPS decoding at 1080p on RTX 5060 while maintaining comparable RD performance to VTM-LDB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。