分频编码提升3D表面重建质量,解决复杂场景建模难题。
FreBIS: Frequency-Based Stratification for Neural Implicit Surface Representations
- 按表面频率分层,每层用专用编码器处理
- 多编码器互补特征,显著提升重建精度
- 适合高复杂度场景的3D建模与渲染
神经隐式表面表示技术在增强现实、虚拟现实、数字孪生和自动驾驶等领域需求旺盛。这类方法将场景表面建模为连续函数,相比传统体素或点云方法取得显著进展。然而,现有方法通常使用单一编码器同时捕捉低频到高频的表面信息,难以应对具有多样复杂表面的场景。为此,本文提出FreBIS——一种基于频率分层的神经隐式表面表示方法。该方法将场景按表面频率划分为多个层级,每个层级由专用编码器处理,并通过新颖的冗余感知加权模块,促进各编码器特征的互异性和互补性。在挑战性数据集BlendedMVS上的实证评估表明,用FreBIS替换现成神经表面重建方法中的标准编码器,能显著提升3D表面重建质量及任意视角下的渲染保真度。
原文摘要 · Abstract (English)
Neural implicit surface representation techniques are in high demand for advancing technologies in augmented reality/virtual reality, digital twins, autonomous navigation, and many other fields. With their ability to model object surfaces in a scene as a continuous function, such techniques have made remarkable strides recently, especially over classical 3D surface reconstruction methods, such as those that use voxels or point clouds. However, these methods struggle with scenes that have varied and complex surfaces principally because they model any given scene with a single encoder network that is tasked to capture all of low through high-surface frequency information in the scene simultaneously. In this work, we propose a novel, neural implicit surface representation approach called FreBIS to overcome this challenge. FreBIS works by stratifying the scene based on the frequency of surfaces into multiple frequency levels, with each level (or a group of levels) encoded by a dedicated encoder. Moreover, FreBIS encourages these encoders to capture complementary information by promoting mutual dissimilarity of the encoded features via a novel, redundancy-aware weighting module. Empirical evaluations on the challenging BlendedMVS dataset indicate that replacing the standard encoder in an off-the-shelf neural surface reconstruction method with our frequency-stratified encoders yields significant improvements. These enhancements are evident both in the quality of the reconstructed 3D surfaces and in the fidelity of their renderings from any viewpoint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。