提出混合精度方案,让神经视频编码在跨平台流媒体中稳定运行。
Streamable Neural Video Compression: A Mixed Precision Approach for Cross-Platform Deployment
- P帧用硬件加速的FP16实时处理,I帧和关键节点强制使用FP32确保同步
- 在12种不同显卡上实现跨代无缝解码,1080p下压缩效率几乎无损失
- 适合需要跨设备稳定部署的5G、卫星直连等实际流媒体场景
神经视频编解码器(NVC)在速率-失真性能上具有突破性优势,适用于5G蜂窝网络和新兴的卫星直连(D2C)链路等带宽受限环境。然而,真实流媒体应用中部署NVC受到跨平台浮点非确定性严重阻碍,导致算术熵编码器在不同GPU架构间不同步甚至崩溃。尽管近期整数量化方法有所缓解,但要么大幅降低压缩效率(INT8),要么因绕过硬件加速造成严重计算瓶颈(INT16)。本文提出一种可流式传输的客户端-服务器NVC架构,采用新颖的混合精度(FP16/FP32)策略:通过在硬件加速的FP16中执行P帧以保障实时吞吐,同时强制I帧和周期性特征适配器重置使用符合IEEE-754标准的FP32,从而在关键边界保证确定性同步。通过对12种覆盖四个架构代际的GPU进行广泛的跨编码/解码测试,证明该方法成功消除同代碎片化问题,并显著提升跨芯片互操作性,在1080p下实现近期架构间的无缝跨代解码。关键的是,这一效果对压缩效率影响微乎其微。此外,我们在多种真实网络环境(包括Wi-Fi 6、5G NR FDD/TDD及Starlink D2C)中评估了系统端到端延迟,验证了可流式学习视频压缩的实际可行性,同时揭示了非地面网络中的独特挑战。
原文摘要 · Abstract (English)
Neural Video Codecs (NVCs) offer unprecedented rate-distortion performance, making them highly attractive for bandwidth-constrained environments like 5G cellular networks and emerging satellite direct-to-cell (D2C) links. However, deploying NVCs in real-world streaming applications is severely hindered by cross-platform floating-point non-determinism, which causes arithmetic entropy coders to desynchronize and crash across different GPU architectures. While recent integer-based quantization methods address this, they incur either massive degradation in compression efficiency (INT8) or severe computational bottlenecks by bypassing hardware acceleration (INT16). In this paper, we propose a streamable, client-server NVC architecture featuring a novel Mixed Precision (FP16/FP32) strategy. By strategically executing P-frames in hardware-accelerated FP16 for real-time throughput, while forcing I-frames and periodic feature-adapter resets to IEEE-754 compliant FP32, we guarantee deterministic synchronization at critical boundaries. Through extensive cross-encode/decode evaluations across 12 GPUs spanning four architectural generations, we demonstrate that our approach successfully eliminates intra-generation fragmentation and substantially broadens cross-die interoperability, achieving seamless cross-generation decodability for recent architectures at 1080p. Crucially, this is achieved with a negligible impact on compression efficiency. Furthermore, we evaluate the system's end-to-end latency across diverse real-world networks, including Wi-Fi 6, 5G NR (FDD/TDD), and Starlink D2C, proving the practical viability of streamable learned video compression while highlighting unique challenges in Non-Terrestrial Networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。