用查询卷积提升高斯点云渲染质量，超越Zip-NeRF
Towards High-Fidelity Gaussian Splatting with Queried-Convolution Neural Networks
- 引入查询卷积，利用坐标与邻域信息增强信号学习
- 在真实场景上实现当前最优的视图合成效果，图像保真度更高
- 适用于多任务，尤其适合追求高质量重建的研究者
高斯点云渲染已革新了新视角合成（NVS）领域，具备更快训练速度和实时渲染能力。然而其重建保真度仍不及强大的辐射场模型如Zip-NeRF。基于理论分析：查询（如坐标）与邻域信息对高保真信号学习至关重要，本文提出查询卷积（Qonvolutions），一种简单而有效的改进方法，利用卷积的邻域特性，将低保真信号与查询结合，输出残差以实现高保真重建。实验证明，将高斯点云与查询卷积神经网络（QNNs）结合，在真实场景上的新视角合成达到当前最优性能，甚至在图像保真度上超越Zip-NeRF。QNNs还提升了1D回归、2D回归与2D超分辨率任务的表现。
原文摘要 · Abstract (English)
Gaussian Splatting has revolutionized the field of Novel View Synthesis (NVS) with faster training and real-time rendering. However, its reconstruction fidelity still trails behind the powerful radiance models such as Zip-NeRF. Motivated by our theoretical result that both queries (such as coordinates) and neighborhood are important to learn high-fidelity signals, this paper proposes Queried-Convolutions (Qonvolutions), a simple yet powerful modification using the neighborhood properties of convolution. Qonvolutions convolve a low-fidelity signal with queries to output residual and achieve high-fidelity reconstruction. We empirically demonstrate that combining Gaussian splatting with Qonvolution neural networks (QNNs) results in state-of-the-art NVS on real-world scenes, even outperforming Zip-NeRF on image fidelity. QNNs also enhance performance of 1D regression, 2D regression and 2D super-resolution tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。