arXiv:2507.04147cs.GRcs.CV2025-07被引 3

用眼动追踪优化3D高斯点云渲染,让虚拟现实更流畅

A3FR: Agile 3D Gaussian Splatting with Incremental Gaze Tracked Foveated Rendering in Virtual Reality

  • 将眼动追踪与渲染并行处理,降低延迟
  • 端到端延迟最高降低2倍,画质保持不变
  • 适合追求低延迟的VR渲染开发人员

虚拟现实显著提升沉浸式数字体验,广泛应用于教育、专业领域和娱乐。图像渲染是关键环节,但3D高斯点云渲染计算开销大,导致实时渲染延迟显著影响用户体验。同时,头戴式显示设备与人类视觉行为紧密相关,利用感知认知知识发展出基于眼动追踪的局部高清渲染技术(即注视区域渲染),可有效降低计算负担。然而,眼动追踪本身也带来额外延迟,有时甚至超过渲染节省。为此,本文提出高效渲染框架A3FR,通过并行化眼动追踪与注视区域渲染过程,最小化整体延迟。采用先进的3D高斯点云渲染算法,实验表明,A3FR可使端到端渲染延迟最高降低2倍,同时保持视觉质量。

原文摘要 · Abstract (English)

Virtual reality (VR) significantly transforms immersive digital interfaces, greatly enhancing education, professional practices, and entertainment by increasing user engagement and opening up new possibilities in various industries. Among its numerous applications, image rendering is crucial. Nevertheless, rendering methodologies like 3D Gaussian Splatting impose high computational demands, driven predominantly by user expectations for superior visual quality. This results in notable processing delays for real-time image rendering, which greatly affects the user experience. Additionally, VR devices such as head-mounted displays (HMDs) are intricately linked to human visual behavior, leveraging knowledge from perception and cognition to improve user experience. These insights have spurred the development of foveated rendering, a technique that dynamically adjusts rendering resolution based on the user's gaze direction. The resultant solution, known as gaze-tracked foveated rendering, significantly reduces the computational burden of the rendering process. Although gaze-tracked foveated rendering can reduce rendering costs, the computational overhead of the gaze tracking process itself can sometimes outweigh the rendering savings, leading to increased processing latency. To address this issue, we propose an efficient rendering framework called~\textit{A3FR}, designed to minimize the latency of gaze-tracked foveated rendering via the parallelization of gaze tracking and foveated rendering processes. For the rendering algorithm, we utilize 3D Gaussian Splatting, a state-of-the-art neural rendering technique. Evaluation results demonstrate that A3FR can reduce end-to-end rendering latency by up to $2\times$ while maintaining visual quality.

VR渲染3D高斯眼动追踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。