arXiv:2602.17120eess.IVcs.MM2026-02

用生成模型提升视频画质,同时保持手机端实时解码

HybridPrompt: Bridging Generative Priors and Traditional Codecs for Mobile Streaming

  • 关键帧用生成模型,其他帧用传统编码,混合架构提速
  • 1080p下解码超150帧/秒,200kbps时比传统编码高8%画质
  • 首次实现生成式视频在手机端实时运行,适合移动端流媒体

在点播场景中,传统编码器因解码效率高而成为行业标准,但在低带宽下质量严重下降。新兴的神经生成编码器虽能显著提升主观画质,但依赖逐帧生成,难以在移动设备上实现实时播放。本文提出 HybridPrompt,首个基于生成模型的视频系统,可在商用智能手机上实现1080p实时解码,速度超过150 FPS。具体而言,采用混合架构:关键帧由生成模型编码,其余帧使用传统编码器。主要挑战在于生成模型的“幻觉”细节与传统编码器的预测机制不匹配,导致码率效率低下。我们证明传统解码过程可微,从而建立端到端优化流程,利用后续帧作为额外监督信号,迫使生成模型生成既高感知质量又数学最优的参考帧。通过两阶段生成策略,系统在速度上远超纯神经基线,同时在200kbps下平均LPIPS提升8%,优于传统编码器。

原文摘要 · Abstract (English)

In Video on Demand (VoD) scenarios, traditional codecs are the industry standard due to their high decoding efficiency. However, they suffer from severe quality degradation under low bandwidth conditions. While emerging generative neural codecs offer significantly higher perceptual quality, their reliance on heavy frame-by-frame generation makes real-time playback on mobile devices impractical. We ask: is it possible to combine the blazing-fast speed of traditional standards with the superior visual fidelity of neural approaches? We present HybridPrompt, the first generative-based video system capable of achieving real-time 1080p decoding at over 150 FPS on a commercial smartphone. Specifically, we employ a hybrid architecture that encodes Keyframes using a generative model while relying on traditional codecs for the remaining frames. A major challenge is that the two paradigms have conflicting objectives: the "hallucinated" details from generative models often misalign with the rigid prediction mechanisms of traditional codecs, causing bitrate inefficiency. To address this, we demonstrate that the traditional decoding process is differentiable, enabling an end-to-end optimization loop. This allows us to use subsequent frames as additional supervision, forcing the generative model to synthesize keyframes that are not only perceptually high-fidelity but also mathematically optimal references for the traditional codec. By integrating a two-stage generation strategy, our system outperforms pure neural baselines by orders of magnitude in speed while achieving an average LPIPS gain of 8% over traditional codecs at 200kbps.

视频编码生成模型移动端混合架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。