低带宽下高效视频传输,靠分层令牌流实现稳定清晰画质。
Resi-VidTok: An Efficient and Decomposed Progressive Tokenization Framework for Ultra-Low-Rate and Lightweight Video Transmission
- 将视频转为重要性排序的令牌流,支持逐步解码和丢包恢复。
- 在0.0004带宽比下仍保持视觉与语义一致,实现实时30帧以上重建。
- 适合对延迟、能耗、可靠性要求高的无线场景使用。
实时无线视频传输在带宽受限和连接弱的条件下仍具挑战性,即使采用先进深度模型亦然。本文提出Resi-VidTok,一种针对超低码率和轻量级视频传输的韧性令牌化框架,在普通数字硬件上实现强鲁棒性,同时保留感知与语义保真度。通过将时空内容重组为离散、按重要性排序的令牌流(含关键令牌与细化令牌),实现渐进编码、前缀可解码重建及受控质量退化。核心贡献是集成差分时间令牌编码的1D韧性令牌化流水线,仅用单一帧级解码器即可可靠恢复不完整令牌集,无需辅助时序提取器或重型生成模型。结合步长控制的帧稀疏化与轻量级解码端插值器,在降低传输负载的同时维持运动连续性。最后,基于通道自适应的源-信道编码与调制方案根据令牌重要性和信道状态动态分配速率与保护,确保在恶劣信噪比下质量稳定。评估结果显示,在通道带宽比(CBR)低至0.0004时仍具备鲁棒的视觉与语义一致性,且实现实时重建超过30 fps,证明其在能效、低延迟、高可靠性无线应用中的实用性。
原文摘要 · Abstract (English)
Real-time transmission of video over wireless networks remains highly challenging, even with advanced deep models, particularly under severe channel conditions such as limited bandwidth and weak connectivity. In this paper, we propose Resi-VidTok, a Resilient Tokenization-Enabled framework designed for ultra-low-rate and lightweight video transmission that delivers strong robustness while preserving perceptual and semantic fidelity on commodity digital hardware. By reorganizing spatio--temporal content into a discrete, importance-ordered token stream composed of key tokens and refinement tokens, Resi-VidTok enables progressive encoding, prefix-decodable reconstruction, and graceful quality degradation under constrained channels. A key contribution is a resilient 1D tokenization pipeline for video that integrates differential temporal token coding, explicitly supporting reliable recovery from incomplete token sets using a single shared framewise decoder--without auxiliary temporal extractors or heavy generative models. Furthermore, stride-controlled frame sparsification combined with a lightweight decoder-side interpolator reduces transmission load while maintaining motion continuity. Finally, a channel-adaptive source--channel coding and modulation scheme dynamically allocates rate and protection according to token importance and channel condition, yielding stable quality across adverse SNRs. Evaluation results indicate robust visual and semantic consistency at channel bandwidth ratios (CBR) as low as 0.0004 and real-time reconstruction at over 30 fps, demonstrating the practicality of Resi-VidTok for energy-efficient, latency-sensitive, and reliability-critical wireless applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。