根据内容和运动动态调整帧率与分辨率,提升移动设备的渲染画质。
Streaming of rendered content with adaptive frame rate and resolution

- 用轻量神经网络预测最优帧率与分辨率组合
- 在带宽受限下提升感知画质并降低计算成本
- 无需修改现有渲染架构,适合移动端实时流媒体
将渲染内容进行流式传输是将高质量图形带给缺乏渲染能力的数十亿移动设备的有效方式。现有方案在服务器端以固定帧率(通常为30或60帧/秒)渲染,并在带宽受限时降低分辨率,但这种策略在带宽约束下导致画质不佳。本文利用人类视觉系统的时空感知极限,通过根据场景内容和运动速度自适应调整帧率与分辨率,从而在降低渲染成本的同时提升主观画质。所提方法不依赖特定编码器,仅需对现有渲染基础设施做最小改动。系统采用轻量级神经网络,基于大量标注了感知视频质量指标的渲染内容数据集进行训练,以预测给定带宽、内容和运动速度下的最优帧率与分辨率组合。实验表明,该方法显著提升感知质量并最小化计算开销。数据集及相关信息详见项目网页:https://www.cl.cam.ac.uk/research/rainbow/projects/adaptive_streaming/。
原文摘要 · Abstract (English)
Streaming rendered content is an attractive way to bring high-quality graphics to billions of mobile devices that do not have sufficient rendering power. Existing solutions render content on a server at a fixed frame rate, typically 30 or 60 frames per second, and reduce resolution when bandwidth is restricted. However, this strategy leads to suboptimal rendering quality under the bandwidth constraints. In this work, we exploit the spatio-temporal limits of the human visual system to improve perceived quality while reducing rendering costs by adaptively adjusting both frame rate and resolution based on scene content and motion. Our approach is codec-agnostic and requires only minimal modifications to existing rendering infrastructure. We propose a system in which a lightweight neural network predicts the optimal combination of frame rate and resolution for a given transmission bandwidth, content, and motion velocity. This prediction significantly enhances perceptual quality while minimizing computational cost under bandwidth constraints. The network is trained on a large dataset of rendered content labeled with a perceptual video quality metric. The dataset and further information can be found at the project web page: https://www.cl.cam.ac.uk/research/rainbow/projects/adaptive_streaming/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。