arXiv:2409.05617cs.CV2024-09被引 1

用少量参数实现高效新视角合成,突破传统NeLF训练难题

G-NeLF: Memory- and Data-Efficient Hybrid Neural Light Field for Novel View Synthesis

  • 基于空间感知特征的网格表示,减少参数量
  • 仅0.95MB模型大小即超越现有NeLF方法性能
  • 参数量仅为Instant-NGP的十分之一,效果更优

受隐式神经表示兴起推动,神经光场(NeLF)被提出以直接预测光线颜色。与神经辐射场(NeRF)不同,NeLF不为每个空间点预测颜色和体密度。然而,现有NeLF方法需先训练一个NeRF模型,并合成超10,000个视图来训练NeLF以提升性能,且渲染质量低于NeRF。本文提出G-NeLF,一种灵活的基于网格的NeLF方法,利用空间感知特征释放神经网络推理潜力,克服训练难题。具体而言,采用精心设计网格生成的空间感知特征序列作为光线表示。基于对多分辨率哈希表适应性的实证研究,提出新型网格化光线表示,以极少参数覆盖全空间。为更好利用序列特征,设计轻量级颜色解码器,模拟光线传播过程,实现更高效推理。G-NeLF无需大量存储开销,模型仅0.95MB即可超越现有最优NeLF;相比网格化NeRF方法如Instant-NGP,仅用其十分之一参数即达更高性能。

原文摘要 · Abstract (English)

Following the burgeoning interest in implicit neural representation, Neural Light Field (NeLF) has been introduced to predict the color of a ray directly. Unlike Neural Radiance Field (NeRF), NeLF does not create a point-wise representation by predicting color and volume density for each point in space. However, the current NeLF methods face a challenge as they need to train a NeRF model first and then synthesize over 10K views to train NeLF for improved performance. Additionally, the rendering quality of NeLF methods is lower compared to NeRF methods. In this paper, we propose G-NeLF, a versatile grid-based NeLF approach that utilizes spatial-aware features to unleash the potential of the neural network's inference capability, and consequently overcome the difficulties of NeLF training. Specifically, we employ a spatial-aware feature sequence derived from a meticulously crafted grid as the ray's representation. Drawing from our empirical studies on the adaptability of multi-resolution hash tables, we introduce a novel grid-based ray representation for NeLF that can represent the entire space with a very limited number of parameters. To better utilize the sequence feature, we design a lightweight ray color decoder that simulates the ray propagation process, enabling a more efficient inference of the ray's color. G-NeLF can be trained without necessitating significant storage overhead and with the model size of only 0.95 MB to surpass previous state-of-the-art NeLF. Moreover, compared with grid-based NeRF methods, e.g., Instant-NGP, we only utilize one-tenth of its parameters to achieve higher performance. Our code will be released upon acceptance.

新视角合成神经光场高效模型网格表示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。