让3D高斯点云重建在模糊、低光等差画质下仍保持高质量。
RobustGS: Unified Boosting of Feedforward 3D Gaussian Splatting under Low-Quality Conditions
- 引入通用退化学习模块,自动识别多视角图像中的多种画质问题。
- 通过语义感知状态空间模型修复退化特征并提取跨视角细粒度对应关系。
- 可即插即用提升现有方法鲁棒性,适用于真实复杂场景的3D重建。
前馈式3D高斯点云散射(3DGS)克服了基于优化的3DGS需逐场景优化的局限,实现快速且高质量的重建。然而,现有前馈方法通常假设输入多视角图像为干净高质量。在真实场景中,图像常受噪声、低光或雨天等恶劣条件影响,导致几何误差与重建质量下降。为此,我们提出一种通用高效的多视角特征增强模块RobustGS,显著提升前馈3DGS在多种不良成像条件下的鲁棒性,实现高质量3D重建。该模块可无缝集成至现有预训练流程中,采用即插即用方式增强重建鲁棒性。具体地,提出新型通用退化学习器,从多视角输入中提取多种退化的通用表征与分布,增强对退化的感知能力,提升整体重建质量。同时,设计一种语义感知状态空间模型:首先利用提取的退化表征在特征空间中修复受损输入;再通过语义感知策略聚合不同视图间语义相似信息,实现细粒度跨视角对应关系提取,进一步提升3D表示质量。大量实验表明,该方法以即插即用方式集成至现有方法后,在多种退化类型下均持续达到领先重建质量。
原文摘要 · Abstract (English)
Feedforward 3D Gaussian Splatting (3DGS) overcomes the limitations of optimization-based 3DGS by enabling fast and high-quality reconstruction without the need for per-scene optimization. However, existing feedforward approaches typically assume that input multi-view images are clean and high-quality. In real-world scenarios, images are often captured under challenging conditions such as noise, low light, or rain, resulting in inaccurate geometry and degraded 3D reconstruction. To address these challenges, we propose a general and efficient multi-view feature enhancement module, RobustGS, which substantially improves the robustness of feedforward 3DGS methods under various adverse imaging conditions, enabling high-quality 3D reconstruction. The RobustGS module can be seamlessly integrated into existing pretrained pipelines in a plug-and-play manner to enhance reconstruction robustness. Specifically, we introduce a novel component, Generalized Degradation Learner, designed to extract generic representations and distributions of multiple degradations from multi-view inputs, thereby enhancing degradation-awareness and improving the overall quality of 3D reconstruction. In addition, we propose a novel semantic-aware state-space model. It first leverages the extracted degradation representations to enhance corrupted inputs in the feature space. Then, it employs a semantic-aware strategy to aggregate semantically similar information across different views, enabling the extraction of fine-grained cross-view correspondences and further improving the quality of 3D representations. Extensive experiments demonstrate that our approach, when integrated into existing methods in a plug-and-play manner, consistently achieves state-of-the-art reconstruction quality across various types of degradations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。