用神经网络将点云转为椭圆高斯，实现毫秒级实时高质量渲染。
Low Latency Point Cloud Rendering with Learned Splatting
- 通过神经网络从任意点云生成3D椭圆高斯表示。
- 支持动态点云实时渲染，延迟低于10毫秒。
- 无需逐场景优化，适配复杂场景与压缩数据。
点云是重要的三维表示形式，广泛应用于新兴场景。由于点稀疏且分布不规则，高质量渲染困难，常需复杂计算恢复连续表面。同时，为避免视觉不适，运动到像素的延迟必须控制在10毫秒以内。现有方法在质量或速度上难以兼顾。为此，我们提出一种框架，实现交互式、自由视角、高保真点云渲染。训练通用神经网络从任意点云估计3D椭圆高斯，并使用可微分表面点绘（differentiable surface splatting）生成任意视角下的平滑纹理与表面法向。该方法无需逐场景优化，支持动态点云的实时渲染。实验表明，该方案在视觉质量、速度、跨场景泛化性及对压缩伪影的鲁棒性方面均表现优异。代码已开源：https://github.com/huzi96/gaussian-pcloud-render。
原文摘要 · Abstract (English)
Point cloud is a critical 3D representation with many emerging applications. Because of the point sparsity and irregularity, high-quality rendering of point clouds is challenging and often requires complex computations to recover the continuous surface representation. On the other hand, to avoid visual discomfort, the motion-to-photon latency has to be very short, under 10 ms. Existing rendering solutions lack in either quality or speed. To tackle these challenges, we present a framework that unlocks interactive, free-viewing and high-fidelity point cloud rendering. We train a generic neural network to estimate 3D elliptical Gaussians from arbitrary point clouds and use differentiable surface splatting to render smooth texture and surface normal for arbitrary views. Our approach does not require per-scene optimization, and enable real-time rendering of dynamic point cloud. Experimental results demonstrate the proposed solution enjoys superior visual quality and speed, as well as generalizability to different scene content and robustness to compression artifacts. The code is available at https://github.com/huzi96/gaussian-pcloud-render .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。