VRSplat解决VR中3D高斯点云的闪烁、漂浮和卡顿问题,实现实时流畅渲染。
VRSplat: Fast and Robust Gaussian Splatting for Virtual Reality
- 融合多技术改进渲染流程,优化核心光栅化器以减少动态误差。
- 在25人用户测试中实现72+帧率,彻底消除画面跳变与视差漂浮。
- 适合开发沉浸式VR应用的团队,尤其关注画质与性能平衡者。
3D高斯点云(3DGS)已成为新视角合成的主流技术,凭借高效的基于软件的GPU光栅化实现卓越性能,适用于实时应用,包括移动设备和低功耗设备。然而,在虚拟现实(VR)中面临三大挑战:(1)时间伪影,如头部移动时出现的画面跳变;(2)投影导致的畸变,引发令人不适且视角不一致的漂浮物;(3)大量高斯点渲染时帧率下降,低于VR所需的临界阈值。相比桌面环境,头戴显示器(HMD)的大视场角、持续头部运动和高分辨率使这些问题加剧。本文提出VRSplat:通过整合并扩展近期多项3DGS进展,系统性应对VR挑战。我们证明了小规模点云(Mini-Splatting)、防跳变(StopThePop)与最优投影(Optimal Projection)可互补协同,通过改进各技术及核心3DGS光栅化器实现。此外,设计了一种高效中央聚焦光栅化器,单次GPU调用处理聚焦区与周边区域,避免冗余计算,提升利用率。方法还包含一个微调步骤,基于StopThePop深度评估和最优投影优化高斯参数。通过25名参与者控制实验验证,结果显示用户强烈偏好VRSplat,优于其他小型点云配置。VRSplat是首个经过系统评估、支持现代VR应用的3DGS方法,在保持72+帧率的同时消除跳变与立体破坏性漂浮物。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has rapidly become a leading technique for novel-view synthesis, providing exceptional performance through efficient software-based GPU rasterization. Its versatility enables real-time applications, including on mobile and lower-powered devices. However, 3DGS faces key challenges in virtual reality (VR): (1) temporal artifacts, such as popping during head movements, (2) projection-based distortions that result in disturbing and view-inconsistent floaters, and (3) reduced framerates when rendering large numbers of Gaussians, falling below the critical threshold for VR. Compared to desktop environments, these issues are drastically amplified by large field-of-view, constant head movements, and high resolution of head-mounted displays (HMDs). In this work, we introduce VRSplat: we combine and extend several recent advancements in 3DGS to address challenges of VR holistically. We show how the ideas of Mini-Splatting, StopThePop, and Optimal Projection can complement each other, by modifying the individual techniques and core 3DGS rasterizer. Additionally, we propose an efficient foveated rasterizer that handles focus and peripheral areas in a single GPU launch, avoiding redundant computations and improving GPU utilization. Our method also incorporates a fine-tuning step that optimizes Gaussian parameters based on StopThePop depth evaluations and Optimal Projection. We validate our method through a controlled user study with 25 participants, showing a strong preference for VRSplat over other configurations of Mini-Splatting. VRSplat is the first, systematically evaluated 3DGS approach capable of supporting modern VR applications, achieving 72+ FPS while eliminating popping and stereo-disrupting floaters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。