arXiv:2505.02005cs.CV2025-05TPAMI被引 5

用专家混合模型高效建模超大规模场景的异质细节,兼顾精度与速度。

Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural Radiance Fields

  • 引入异质哈希专家网络,自动分解场景并分配专属神经辐射场。
  • 在大于6.5平方公里的真实场景上实现端到端训练,渲染提速16倍。
  • 适合需要高保真大场景重建的研究者与工业应用开发者。

大规模场景的神经辐射场(NeRF)方法强调场景分解对可扩展性的重要性。尽管已实现合理可扩展性,但可学习分解、场景异质性建模和效率问题仍待解决。本文提出Switch-NeRF++,一种异质哈希专家混合(HMoHE)网络,在统一框架中解决上述挑战。该模型通过门控网络自动将3D点分配给专用的NeRF专家,并通过提出的稀疏门控专家混合(MoE)NeRF框架与专家协同优化。采用基于哈希的门控网络与不同分辨率范围的异质哈希专家,高效学习大规模场景的分解结构。这些设计使系统在真实世界大场景建模中兼具高质量与高效率。我们在现有大场景数据集及一个来自UrbanBIS的新数据集(>6.5km²)上评估了准确性与可扩展性。实验表明,本方法可轻松扩展至多种大场景,达到当前最佳渲染精度;同时训练加速8倍,渲染加速16倍,显著优于Switch-NeRF。代码将在https://github.com/MiZhenxing/Switch-NeRF发布。

原文摘要 · Abstract (English)

Recent NeRF methods on large-scale scenes have underlined the importance of scene decomposition for scalable NeRFs. Although achieving reasonable scalability, there are several critical problems remaining unexplored, i.e., learnable decomposition, modeling scene heterogeneity, and modeling efficiency. In this paper, we introduce Switch-NeRF++, a Heterogeneous Mixture of Hash Experts (HMoHE) network that addresses these challenges within a unified framework. It is a highly scalable NeRF that learns heterogeneous decomposition and heterogeneous NeRFs efficiently for large-scale scenes in an end-to-end manner. In our framework, a gating network learns to decompose scenes and allocates 3D points to specialized NeRF experts. This gating network is co-optimized with the experts by our proposed Sparsely Gated Mixture of Experts (MoE) NeRF framework. We incorporate a hash-based gating network and distinct heterogeneous hash experts. The hash-based gating efficiently learns the decomposition of the large-scale scene. The distinct heterogeneous hash experts consist of hash grids of different resolution ranges, enabling effective learning of the heterogeneous representation of different scene parts. These design choices make our framework an end-to-end and highly scalable NeRF solution for real-world large-scale scene modeling to achieve both quality and efficiency. We evaluate our accuracy and scalability on existing large-scale NeRF datasets and a new dataset with very large-scale scenes ($>6.5km^2$) from UrbanBIS. Extensive experiments demonstrate that our approach can be easily scaled to various large-scale scenes and achieve state-of-the-art scene rendering accuracy. Furthermore, our method exhibits significant efficiency, with an 8x acceleration in training and a 16x acceleration in rendering compared to Switch-NeRF. Codes will be released at https://github.com/MiZhenxing/Switch-NeRF.

NeRF大场景建模专家混合高效渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。