arXiv:2512.24547cs.CV2025-12被引 3

用分层向量量化生成低分辨率视频的紧凑高保真表示,适合边缘设备部署。

Hierarchical Vector-Quantized Latents for Perceptual Low-Resolution Video Compression

  • 设计多尺度分层向量量化自编码器,3D残差卷积构建时空特征。
  • 64x64视频在测试集达25.96 dB PSNR、0.8375 SSIM,比单尺度提升1.41 dB。
  • 轻量模型仅1850万参数,适合移动设备与带宽受限场景使用。

视频流量激增对带宽和存储基础设施提出更高要求,尤其在内容分发网络(CDNs)和边缘设备中。传统视频编码器如H.264和HEVC虽压缩率高,但以像素域重建为主,缺乏对机器学习潜空间表示的原生支持,难以融入深度学习流程。本文提出多尺度向量量化变分自编码器(MS-VQ-VAE),旨在生成适用于高效存储、传输与客户端解码的低分辨率视频紧凑高保真潜表示。模型扩展VQ-VAE-2至时空域,采用3D残差卷积构建两级分层潜结构。模型轻量(约1850万参数),专为64×64分辨率视频片段优化,适合计算与内存受限的边缘设备。为提升感知重建质量,引入基于预训练VGG16的感知损失。在UCF101数据集上使用2秒视频片段(32帧,16 FPS)训练,在测试集上达到25.96 dB PSNR和0.8375 SSIM;验证时相较单尺度基线提升1.41 dB PSNR与0.0248 SSIM。该框架适用于实时流媒体、移动端视频分析及CDN级存储优化等带宽敏感场景。

原文摘要 · Abstract (English)

The exponential growth of video traffic has placed increasing demands on bandwidth and storage infrastructure, particularly for content delivery networks (CDNs) and edge devices. While traditional video codecs like H.264 and HEVC achieve high compression ratios, they are designed primarily for pixel-domain reconstruction and lack native support for machine learning-centric latent representations, limiting their integration into deep learning pipelines. In this work, we present a Multi-Scale Vector Quantized Variational Autoencoder (MS-VQ-VAE) designed to generate compact, high-fidelity latent representations of low-resolution video, suitable for efficient storage, transmission, and client-side decoding. Our architecture extends the VQ-VAE-2 framework to a spatiotemporal setting, introducing a two-level hierarchical latent structure built with 3D residual convolutions. The model is lightweight (approximately 18.5M parameters) and optimized for 64x64 resolution video clips, making it appropriate for deployment on edge devices with constrained compute and memory resources. To improve perceptual reconstruction quality, we incorporate a perceptual loss derived from a pre-trained VGG16 network. Trained on the UCF101 dataset using 2-second video clips (32 frames at 16 FPS), on the test set we achieve 25.96 dB PSNR and 0.8375 SSIM. On validation, our model improves over the single-scale baseline by 1.41 dB PSNR and 0.0248 SSIM. The proposed framework is well-suited for scalable video compression in bandwidth-sensitive scenarios, including real-time streaming, mobile video analytics, and CDN-level storage optimization.

视频压缩向量量化边缘计算潜空间建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。