用神经辐射场融合多光谱数据,提升3D重建精度与真实感。
Multispectral-NeRF:a multispectral modeling approach based on neural radiance fields
- 基于NeRF扩展6通道光谱输入,增强多光谱信息建模能力。
- 通过重设计残差函数,显著降低重建与参考图像的光谱差异。
- 适配高比特深度压缩模块,解决多光谱图像存储与计算挑战。
3D重建技术利用2D图像等传感器数据生成真实物体、场景或环境的三维表示,在机器人、自动驾驶和虚拟现实等领域有广泛应用。传统基于2D图像的3D重建方法通常仅依赖RGB光谱信息。随着传感器技术进步,更多非RGB光谱波段被引入重建流程。现有融合多光谱数据的方法普遍存在成本高、精度低、几何特征差等问题。基于NeRF的3D重建可有效解决当前多光谱重建中的多项挑战,实现高精度高质量重建。然而,现有NeRF及其改进模型(如NeRFacto)仅在三通道数据上训练,无法充分处理多波段信息。为此,本文提出Multispectral-NeRF,一种从NeRF衍生的增强神经架构,能有效整合多光谱信息。技术贡献包括三方面:扩大隐藏层维度以支持6波段光谱输入;重设计残差函数以优化重建图像与参考图像间的光谱差异计算;适配数据压缩模块以应对多光谱影像更高的比特深度需求。实验结果表明,Multispectral-NeRF成功处理多波段光谱特征,并准确保留原始场景的光谱特性。
原文摘要 · Abstract (English)
3D reconstruction technology generates three-dimensional representations of real-world objects, scenes, or environments using sensor data such as 2D images, with extensive applications in robotics, autonomous vehicles, and virtual reality systems. Traditional 3D reconstruction techniques based on 2D images typically relies on RGB spectral information. With advances in sensor technology, additional spectral bands beyond RGB have been increasingly incorporated into 3D reconstruction workflows. Existing methods that integrate these expanded spectral data often suffer from expensive scheme prices, low accuracy and poor geometric features. Three - dimensional reconstruction based on NeRF can effectively address the various issues in current multispectral 3D reconstruction methods, producing high - precision and high - quality reconstruction results. However, currently, NeRF and some improved models such as NeRFacto are trained on three - band data and cannot take into account the multi - band information. To address this problem, we propose Multispectral-NeRF, an enhanced neural architecture derived from NeRF that can effectively integrates multispectral information. Our technical contributions comprise threefold modifications: Expanding hidden layer dimensionality to accommodate 6-band spectral inputs; Redesigning residual functions to optimize spectral discrepancy calculations between reconstructed and reference images; Adapting data compression modules to address the increased bit-depth requirements of multispectral imagery. Experimental results confirm that Multispectral-NeRF successfully processes multi-band spectral features while accurately preserving the original scenes' spectral characteristics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。