用MCTS动态调整采样粒度,让神经辐射场训练快2.3倍
MCBlock: Boosting Neural Radiance Field Training Speed by MCTS-based Dynamic-Resolution Ray Sampling
- 基于MCTS自动划分图像块,不同纹理区域用不同分辨率采样
- 在Nerfstudio上实现最高2.33倍训练加速,且不降低画质
- 适合需要快速渲染的医疗、实时视频等多媒体应用
神经辐射场(NeRF)虽能生成高保真新视角图像,但其训练速度慢,即使最先进的高斯泼溅模型也需数分钟,难以满足远程医疗等多媒体场景的实时需求。现有采样方法对简单与复杂纹理区域均采用统一采样策略,或全图以像素级精细采样,造成显卡资源浪费。实际上,不同纹理强度区域应匹配不同采样粒度。为此,本文提出基于蒙特卡洛树搜索(MCTS)的动态分辨率射线采样算法MCBlock,将训练图像划分为大小不同的像素块,实现分块主动训练。树结构初始时依据图像纹理快速构建,扩展/剪枝模块动态优化块划分。该方法在开源工具Nerfstudio中实现,训练速度最高提升2.33倍,优于其他采样算法,可适配任意锥追踪型NeRF模型,助力多媒体领域应用。
原文摘要 · Abstract (English)
Neural Radiance Field (NeRF) is widely known for high-fidelity novel view synthesis. However, even the state-of-the-art NeRF model, Gaussian Splatting, requires minutes for training, far from the real-time performance required by multimedia scenarios like telemedicine. One of the obstacles is its inefficient sampling, which is only partially addressed by existing works. Existing point-sampling algorithms uniformly sample simple-texture regions (easy to fit) and complex-texture regions (hard to fit), while existing ray-sampling algorithms sample these regions all in the finest granularity (i.e. the pixel level), both wasting GPU training resources. Actually, regions with different texture intensities require different sampling granularities. To this end, we propose a novel dynamic-resolution ray-sampling algorithm, MCBlock, which employs Monte Carlo Tree Search (MCTS) to partition each training image into pixel blocks with different sizes for active block-wise training. Specifically, the trees are initialized according to the texture of training images to boost the initialization speed, and an expansion/pruning module dynamically optimizes the block partition. MCBlock is implemented in Nerfstudio, an open-source toolset, and achieves a training acceleration of up to 2.33x, surpassing other ray-sampling algorithms. We believe MCBlock can apply to any cone-tracing NeRF model and contribute to the multimedia community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。