基于实时注视点动态调整画质与码率，提升VR云游戏流畅度与视觉体验。
EyeNexus: Adaptive Gaze-Driven Quality and Bitrate Streaming for Seamless VR Cloud Gaming Experiences
- 根据用户注视点和网络带宽实时调整画面压缩与编码策略。
- 在不同网络条件下降低70.9%延迟，感知画质提升24.6%。
- 适合追求高沉浸感的VR游戏玩家与云游戏平台开发者。
虚拟现实（VR）云游戏系统将3D图形在云端服务器渲染，以在头戴设备上运行高画质游戏。受限于网络波动，高质量场景传输面临挑战。利用人类视觉感知非均匀性，视点聚焦渲染与编码已被证明可优化受限网络下的流媒体传输。现有先进方法或未融合实时注视数据，或无法适应网络变化，导致用户体验不佳。我们提出EyeNexus，首个结合实时注视驱动空间压缩（FSC）与注视驱动视频编码（FVE）的系统，通过精准对齐注视点实现视点聚焦。设计新型动态视点模型，依据实时带宽与注视数据自适应调整聚焦区域，简化FVE中的网络感知质量分配，确保画质平滑过渡。在多种网络条件与游戏下，通过客观与主观评估验证。结果表明，EyeNexus可降低延迟达70.9%，感知画质提升最高24.6%。经机构伦理审查的用户研究显示，其在可玩性与视觉质量上分别提升高达48%，且消除晕动症。
原文摘要 · Abstract (English)
Virtual Reality (VR) cloud gaming systems render the 3D graphics on cloud servers for playing graphically demanding games on VR headsets. Delivering high-resolution game scenes is challenging due to variation in network performance. By leveraging the non-uniform human vision perception, foveated rendering and encoding have proven effective for optimized streaming in constrained networks. SoTA foveation methods either do not incorporate real-time gaze data or are unable to handle variations in network conditions, resulting in a suboptimal user experience. We introduce EyeNexus, a pioneering system that combines real-time gaze-driven spatial compression (FSC) with gaze-driven video encoding (FVE), transforming the gaze point for precise alignment and foveation. We propose a novel foveation model that dynamically adjusts the foveation region based on real-time bandwidth and gaze data. The model simplifies network-aware quality assignment in FVE, ensuring smooth and imperceptible quality gradients. We evaluate EyeNexus using objective and subjective measures with different network conditions and games. EyeNexus reduces latency by up to 70.9% and improves perceptual visual quality by up to 24.6%. Our IRB-approved user study shows that EyeNexus achieves the highest playability and visual quality, with improvements of up to 48%, while eliminating motion sickness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。