用几何引导视频生成，修复3D高斯溅射的噪点和闪烁问题
GaussFusion: Improving 3D Reconstruction in the Wild with A Geometry-Informed Video Generator
- 通过几何信息指导视频生成，优化3DGS重建结果
- 在新视角合成任务上达到当前最佳性能，实时版15帧/秒
- 适合需要高质量3D交互应用的研究与开发者
我们提出GaussFusion，一种通过几何引导视频生成来提升野外场景下3D高斯溅射（3DGS）重建质量的新方法。该方法缓解了因相机位姿误差、覆盖不全及初始几何噪声导致的浮点物、闪烁和模糊等常见问题。不同于以往仅依赖RGB的单一流程方法，GaussFusion引入一个几何感知的视频到视频生成器，可对基于优化和前馈两种方式的3DGS结果进行统一精炼。给定已有重建，系统渲染包含深度、法向、不透明度和协方差的高斯原始体视频缓冲区，由生成器输出时间连贯且无瑕疵的帧。我们还设计了一套退化模式合成管道，模拟多样化失真，增强模型鲁棒性与泛化能力。GaussFusion在新视角合成基准上取得领先表现，其高效版本可在15 FPS下实时运行，保持相近性能，适用于交互式3D应用。
原文摘要 · Abstract (English)
We present GaussFusion, a novel approach for improving 3D Gaussian splatting (3DGS) reconstructions in the wild through geometry-informed video generation. GaussFusion mitigates common 3DGS artifacts, including floaters, flickering, and blur caused by camera pose errors, incomplete coverage, and noisy geometry initialization. Unlike prior RGB-based approaches limited to a single reconstruction pipeline, our method introduces a geometry-informed video-to-video generator that refines 3DGS renderings across both optimization-based and feed-forward methods. Given an existing reconstruction, we render a Gaussian primitive video buffer encoding depth, normals, opacity, and covariance, which the generator refines to produce temporally coherent, artifact-free frames. We further introduce an artifact synthesis pipeline that simulates diverse degradation patterns, ensuring robustness and generalization. GaussFusion achieves state-of-the-art performance on novel-view synthesis benchmarks, and an efficient variant runs in real time at 15 FPS while maintaining similar performance, enabling interactive 3D applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。