arXiv:2411.05731cs.CV2024-11被引 2

提升3D高斯渲染的视角自适应能力,增强细节与色彩真实感。

PEP-GS: Perceptually-Enhanced Precise Structured 3D Gaussians for View-Adaptive Rendering

  • 动态预测高斯属性,用分层结构注意力替代球谐函数。
  • 在视点依赖效果和细粒度细节上显著优于现有方法。
  • 适合需要高质量真实感渲染的场景建模与视觉应用。

近期,3D高斯泼溅(3D-GS)在实时、高质量三维场景渲染中取得显著进展。然而,仍面临高斯冗余、捕捉视点依赖效应能力有限、复杂光照与镜面反射处理困难等问题。使用球谐函数表示颜色的方法难以有效建模各向异性成分,尤其在复杂光照下导致对比度不足与色彩饱和度失真。为此,我们提出PEP-GS,一种感知增强的框架,可动态预测高斯属性(包括不透明度、颜色和协方差)。通过引入分层颗粒化结构注意力机制,取代传统球谐函数,更精准建模复杂视点依赖色效。采用稳定可解释的不透明度与协方差估计框架,避免过早剔除关键高斯点,保障场景表征准确性。此外,对最终渲染图像进行感知优化,提升不同视图间的感知一致性,增强纹理保真度与细尺度细节保留。实验表明,PEP-GS在视点依赖效果与细粒度细节等挑战性场景中显著超越现有最优方法。

原文摘要 · Abstract (English)

Recently, 3D Gaussian Splatting (3D-GS) has achieved significant success in real-time, high-quality 3D scene rendering. However, it faces several challenges, including Gaussian redundancy, limited ability to capture view-dependent effects, and difficulties in handling complex lighting and specular reflections. Additionally, methods that use spherical harmonics for color representation often struggle to effectively capture anisotropic components, especially when modeling view-dependent colors under complex lighting conditions, leading to insufficient contrast and unnatural color saturation. To address these limitations, we introduce PEP-GS, a perceptually-enhanced framework that dynamically predicts Gaussian attributes, including opacity, color, and covariance. We replace traditional spherical harmonics with a Hierarchical Granular-Structural Attention mechanism, which enables more accurate modeling of complex view-dependent color effects. By employing a stable and interpretable framework for opacity and covariance estimation, PEP-GS avoids the removal of essential Gaussians prematurely, ensuring a more accurate scene representation. Furthermore, perceptual optimization is applied to the final rendered images, enhancing perceptual consistency across different views and ensuring high-quality renderings with improved texture fidelity and fine-scale detail preservation. Experimental results demonstrate that PEP-GS outperforms state-of-the-art methods, particularly in challenging scenarios involving view-dependent effects and fine-scale details.

3D高斯渲染优化视角自适应感知增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。