提升动态场景3D高斯点渲染速度与鲁棒性,兼顾高质量重建。
Fast and Robust Deformable 3D Gaussian Splatting
- 用粗到精时间嵌入融合加速渲染,减少计算开销。
- 在弱初始化下仍可稳定重建,细节保留更完整。
- 解决暗光场景颜色失真问题,适合真实复杂动态场景。
3D高斯点绘图在静态场景的新视角合成中展现出卓越的实时渲染能力与视觉质量。基于此优势,研究者逐步将3D高斯扩展至动态场景重建。基于变形场的方法通过在标准场中保持3D高斯属性,并利用变形场跨时序变换该场,成为一种有前景的方案。然而,现有方法常面临渲染速度不足、对初始点云依赖性强、在暗光场景中易陷入局部最优等问题。为此,我们提出FRoG框架,实现高效且鲁棒的高质量动态场景重建。FRoG结合每个高斯的嵌入与粗到精的时间嵌入策略,通过早期融合时间嵌入加速渲染。为增强稀疏初始化下的鲁棒性,提出一种深度与误差引导采样策略,在低偏差初始位置填充新3D高斯,显著降低变形场优化负担,提升静态与动态区域的细节重建效果。此外,通过调节透明度变化,缓解暗光场景中的局部最优问题,改善色彩保真度。全面实验验证,本方法在保持最先进视觉质量的同时,实现更快的渲染速度。
原文摘要 · Abstract (English)
3D Gaussian Splatting has demonstrated remarkable real-time rendering capabilities and superior visual quality in novel view synthesis for static scenes. Building upon these advantages, researchers have progressively extended 3D Gaussians to dynamic scene reconstruction. Deformation field-based methods have emerged as a promising approach among various techniques. These methods maintain 3D Gaussian attributes in a canonical field and employ the deformation field to transform this field across temporal sequences. Nevertheless, these approaches frequently encounter challenges such as suboptimal rendering speeds, significant dependence on initial point clouds, and vulnerability to local optima in dim scenes. To overcome these limitations, we present FRoG, an efficient and robust framework for high-quality dynamic scene reconstruction. FRoG integrates per-Gaussian embedding with a coarse-to-fine temporal embedding strategy, accelerating rendering through the early fusion of temporal embeddings. Moreover, to enhance robustness against sparse initializations, we introduce a novel depth- and error-guided sampling strategy. This strategy populates the canonical field with new 3D Gaussians at low-deviation initial positions, significantly reducing the optimization burden on the deformation field and improving detail reconstruction in both static and dynamic regions. Furthermore, by modulating opacity variations, we mitigate the local optima problem in dim scenes, improving color fidelity. Comprehensive experimental results validate that our method achieves accelerated rendering speeds while maintaining state-of-the-art visual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。