arXiv:2501.06880cs.NIcs.CV2025-01被引 2

通过复用已训练模型,实现云游戏视频的毫秒级实时画质增强。

Real-Time Neural-Enhancement for Online Cloud Gaming

  • 利用游戏画面重复性,复用预训练超分模型减少重训时间。
  • 实测比当前最优方案提升1.81dB峰值信噪比,冗余训练降低44%。
  • 适合追求低延迟高画质的移动云游戏用户和系统开发者。

在线云游戏需要在多变的广域网环境下实现实时、高质量的视频传输。基于超分辨率(SR)的神经增强传输算法虽能有效应对网络挑战,但需对整段视频进行密集微调,难以适应多样化的在线云游戏场景。为此,我们提出River框架,基于云游戏视频片段特征高度重复与冗余的观察,显著降低微调延迟。该框架构建内容感知编码器,为不同视频片段微调并存储超分模型至查表中;在线传输时,根据视频特征检索最匹配模型以增强画质;若现有模型效果不足,则动态微调新模型并更新查表。为避免向客户端流式传输模型权重,设计预测性预取策略,提前加载高概率被调用的模型。真实游戏流测试表明,River可减少44%冗余训练开销,峰值信噪比提升1.81dB;实际部署满足实时性要求,在移动端实现约720p 20fps流畅体验。

原文摘要 · Abstract (English)

Online Cloud gaming demands real-time, high-quality video transmission across variable wide-area networks (WANs). Neural-enhanced video transmission algorithms employing super-resolution (SR) for video quality enhancement have effectively challenged WAN environments. However, these SR-based methods require intensive fine-tuning for the whole video, making it infeasible in diverse online cloud gaming. To address this, we introduce River, a cloud gaming delivery framework designed based on the observation that video segment features in cloud gaming are typically repetitive and redundant. This permits a significant opportunity to reuse fine-tuned SR models, reducing the fine-tuning latency of minutes to query latency of milliseconds. To enable the idea, we design a practical system that addresses several challenges, such as model organization, online model scheduler, and transfer strategy. River first builds a content-aware encoder that fine-tunes SR models for diverse video segments and stores them in a lookup table. When delivering cloud gaming video streams online, River checks the video features and retrieves the most relevant SR models to enhance the frame quality. Meanwhile, if no existing SR model performs well enough for some video segments, River will further fine-tune new models and update the lookup table. Finally, to avoid the overhead of streaming model weight to the clients, River designs a prefetching strategy that predicts the models with the highest possibility of being retrieved. Our evaluation based on real video game streaming demonstrates River can reduce redundant training overhead by 44% and improve the Peak-Signal-to-Noise-Ratio by 1.81dB compared to the SOTA solutions. Practical deployment shows River meets real-time requirements, achieving approximately 720p 20fps on mobile devices.

云游戏超分辨率实时增强模型复用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。