通过分区域训练局部高斯,实现大场景自由视角的高效高质渲染
Toy-GS: Assembling Local Gaussians for Precisely Rendering Large-Scale Free Camera Trajectories
- 按相机位姿自适应划分场景区域,分块并行训练局部高斯
- 在两个公开数据集和自建SCUTic上实现最高性能,PSNR提升1.19 dB
- 适合需要节省显存的大规模3D渲染任务,尤其支持自由轨迹
当前针对大规模自由相机轨迹的3D渲染面临两大挑战:一是相机分布不规则且场景类型多样;二是处理完整点云和图像需大量GPU内存。本文提出Toy-GS方法,通过自适应空间划分将相机与稀疏点云分至不同区域,对每个区域的局部高斯并行训练,聚焦纹理细节并降低显存占用。进一步引入多视角约束与位置感知点自适应控制(PPAC)提升纹理质量。此外,区域融合策略结合局部与全局高斯,随区域数量增加持续优化渲染效果。大量实验验证了其有效性与效率,在两个公开大规模数据集及自建SCUTic数据集上达到领先性能,相比基准方法提升1.19 dB PSNR,节省7 GB GPU内存。
原文摘要 · Abstract (English)
Currently, 3D rendering for large-scale free camera trajectories, namely, arbitrary input camera trajectories, poses significant challenges: 1) The distribution and observation angles of the cameras are irregular, and various types of scenes are included in the free trajectories; 2) Processing the entire point cloud and all images at once for large-scale scenes requires a substantial amount of GPU memory. This paper presents a Toy-GS method for accurately rendering large-scale free camera trajectories. Specifically, we propose an adaptive spatial division approach for free trajectories to divide cameras and the sparse point cloud of the entire scene into various regions according to camera poses. Training each local Gaussian in parallel for each area enables us to concentrate on texture details and minimize GPU memory usage. Next, we use the multi-view constraint and position-aware point adaptive control (PPAC) to improve the rendering quality of texture details. In addition, our regional fusion approach combines local and global Gaussians to enhance rendering quality with an increasing number of divided areas. Extensive experiments have been carried out to confirm the effectiveness and efficiency of Toy-GS, leading to state-of-the-art results on two public large-scale datasets as well as our SCUTic dataset. Our proposal demonstrates an enhancement of 1.19 dB in PSNR and conserves 7 G of GPU memory when compared to various benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。