MLVC实现跨平台神经视频编码,兼顾高压缩率与实时性。
MLVC: Multi-platform Learned Video Codec for Real-World Deployment

- 通过超先验传递尺度参数,确保不同硬件上解码一致
- 在视频会议数据集上比硬件HEVC节省超70%码率
- 支持苹果/高通/英特尔等主流芯片实时运行,适合落地
神经视频编码器虽在压缩效率上超越传统编码器,但因跨平台不兼容和计算开销高难以部署。现有基于量化的方法无法在不同硬件上产生确定性结果,导致解码失败。我们提出MLVC,一种面向实际部署的硬件鲁棒型神经视频编码器。核心思想是通过超先验显式传输尺度参数,保证设备间熵编码一致性,无需位级精确计算。虽增加码率开销,但通过门控记忆、ReGLU激活、长期参考恢复机制及领域感知训练,有效弥补效率损失。在VCD视频会议基准测试中,MLVC相比硬件HEVC(最强可部署基线)实现>70%的BD-rate(MOS)提升,主观质量媲美无法跨平台运行的DCVC-RT。编码器与解码器在苹果、英特尔、高通的消费级NPU上平均达到100 FPS。MLVC是首个兼具优秀压缩性能、实时速度与跨设备鲁棒性的神经视频编码器,适用于大规模部署。代码已开源:https://github.com/microsoft/mlvc。
原文摘要 · Abstract (English)
Neural video codecs have surpassed classical codecs in coding efficiency but remain impractical for deployment due to cross-platform incompatibility and high computational cost. Existing quantization-based solutions fail to produce deterministic results across diverse hardware platforms, leading to catastrophic decoding failures. We introduce MLVC, a hardware-robust neural video codec designed for practical cross-platform inference. The key idea is to explicitly transmit scale parameters through the hyperprior, which guarantees entropy coding consistency across devices without requiring bit-exact arithmetic. While this increases bitrate overhead, we recover most of the coding efficiency through architectural improvements (gated memory, ReGLU activation), a long-term reference recovery mechanism, and domain-specific perceptual training. On the VCD video conferencing benchmark, MLVC achieves >70% BD-rate (MOS) improvement over hardware HEVC, the strongest deployable baseline, while reaching subjective quality competitive with DCVC-RT, which cannot operate across diverse platforms. Both the encoder and decoder run at 100 FPS on average on commodity NPUs from Apple, Intel, and Qualcomm. MLVC is the first neural video codec to combine competitive compression performance, real-time speed, and cross-platform robustness across diverse consumer devices, making it suitable for widespread deployment. Code is available at https://github.com/microsoft/mlvc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。