从单张压缩图像重建动态3D场景,突破传统方法局限
SCIGS: 3D Gaussians Splatting from a Snapshot Compressive Image
- 用相机位姿和高斯原始体坐标做嵌入,改进3DGS结构
- 首次实现单张压缩图像重建动态3D场景,精度优于现有方法
- 适合高速动态场景重建,对压缩成像领域有实用价值
快照压缩成像(SCI)可捕捉高速动态场景信息,但高效重建方法仍面临挑战:基于深度学习的方法难以保持场景三维结构一致性,而基于NeRF的方法在处理动态场景时仍有局限。为此,我们提出SCIGS,一种3DGS的变体,设计了基于原始体级别的变换网络,利用相机位姿戳记和高斯原始体坐标作为嵌入向量,避免了原生3DGS对相机位姿的依赖,并通过变换后的原始体增强多视角3D结构一致性。同时引入高频滤波器,消除变换过程中的伪影。SCIGS是首个从单张压缩图像重建显式3D场景的方法,可扩展至动态3D场景。在静态与动态场景上的实验表明,该方法不仅提升了SCI解码性能,且在单张压缩图像重建动态3D场景方面优于当前最先进方法。代码将在发表后公开。
原文摘要 · Abstract (English)
Snapshot Compressive Imaging (SCI) offers a possibility for capturing information in high-speed dynamic scenes, requiring efficient reconstruction method to recover scene information. Despite promising results, current deep learning-based and NeRF-based reconstruction methods face challenges: 1) deep learning-based reconstruction methods struggle to maintain 3D structural consistency within scenes, and 2) NeRF-based reconstruction methods still face limitations in handling dynamic scenes. To address these challenges, we propose SCIGS, a variant of 3DGS, and develop a primitive-level transformation network that utilizes camera pose stamps and Gaussian primitive coordinates as embedding vectors. This approach resolves the necessity of camera pose in vanilla 3DGS and enhances multi-view 3D structural consistency in dynamic scenes by utilizing transformed primitives. Additionally, a high-frequency filter is introduced to eliminate the artifacts generated during the transformation. The proposed SCIGS is the first to reconstruct a 3D explicit scene from a single compressed image, extending its application to dynamic 3D scenes. Experiments on both static and dynamic scenes demonstrate that SCIGS not only enhances SCI decoding but also outperforms current state-of-the-art methods in reconstructing dynamic 3D scenes from a single compressed image. The code will be made available upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。