arXiv:2501.13045cs.CVcs.MM2025-01被引 41

用绘画分层思路压缩3D高斯模型,大幅减小体积且保持画质。

Sketch and Patch: Efficient 3D Gaussian Representation for Man-Made Scenes

  • 将高斯分为轮廓和区域两类,分别用参数化与量化优化
  • 同等大小下图像质量提升超45%,室内场景仅需原2.3%体积
  • 适合需要轻量化3D重建的实时应用或移动端部署

3D高斯点阵(3DGS)已成为实现逼真三维场景渲染的有前景方法,但其高存储需求限制了实际应用。我们观察到高斯点具有类似艺术创作中线条与涂色的差异:部分高斯点捕捉边缘等高频特征,类似素描轮廓;另一部分则表示平滑区域,类似于大笔触填充体积与深度。基于此,我们提出一种新型混合表示法,将高斯点分为(i)轮廓高斯点(Sketch Gaussians),用于定义场景边界;(ii)区域高斯点(Patch Gaussians),用于覆盖平滑区域。轮廓高斯点通过参数化模型高效编码,利用其几何一致性;区域高斯点则通过优化剪枝、重训练与向量量化,维持体感一致性和存储效率。在多种室内外场景上的综合评估表明,该结构感知方法在相同模型规模下,PSNR提升达32.62%,SSIM提升19.12%,LPIPS降低45.41%;对于一个室内场景,模型仅需原模型2.3%的大小即可保持视觉质量。

原文摘要 · Abstract (English)

3D Gaussian Splatting (3DGS) has emerged as a promising representation for photorealistic rendering of 3D scenes. However, its high storage requirements pose significant challenges for practical applications. We observe that Gaussians exhibit distinct roles and characteristics that are analogous to traditional artistic techniques -- Like how artists first sketch outlines before filling in broader areas with color, some Gaussians capture high-frequency features like edges and contours; While other Gaussians represent broader, smoother regions, that are analogous to broader brush strokes that add volume and depth to a painting. Based on this observation, we propose a novel hybrid representation that categorizes Gaussians into (i) Sketch Gaussians, which define scene boundaries, and (ii) Patch Gaussians, which cover smooth regions. Sketch Gaussians are efficiently encoded using parametric models, leveraging their geometric coherence, while Patch Gaussians undergo optimized pruning, retraining, and vector quantization to maintain volumetric consistency and storage efficiency. Our comprehensive evaluation across diverse indoor and outdoor scenes demonstrates that this structure-aware approach achieves up to 32.62% improvement in PSNR, 19.12% in SSIM, and 45.41% in LPIPS at equivalent model sizes, and correspondingly, for an indoor scene, our model maintains the visual quality with 2.3% of the original model size.

3D重建高斯点阵压缩渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。