用强化学习选关键视角,减少高质量渲染图像需求。
DNRSelect: Active Best View Selection for Deferred Neural Rendering
- 用强化学习从普通渲染图中选最优视角
- 仅需少量光线追踪图即可实现高保真渲染
- 适合追求高效高质渲染的图形与机器人研究者
延迟神经渲染(DNR)是一种新兴的计算机图形学流程,用于高保真渲染和机器人感知。然而,DNR严重依赖大量光线追踪图像数据集,并需要大量计算资源。如何在减少对高质量光线追踪图像依赖的同时保持渲染质量仍属未充分探索的问题。本文提出DNRSelect,融合基于强化学习的视角选择器与3D纹理聚合器。我们首先设计一种新型强化学习视角选择器,在易获取的光栅化图像上训练,以识别最优视角。仅对这些选定视角采集少量光线追踪图像,即可实现高质量渲染。为增强空间感知与几何一致性,引入3D纹理聚合器,融合深度图、法线图的金字塔特征与UV图。由于光线追踪图像获取耗时远高于光栅化图像,DNRSelect通过仅使用少数精选视角,大幅降低对光线追踪数据的需求,同时保持高保真渲染效果。我们在NeRF-Synthetic数据集上进行了详尽实验与消融研究,验证了DNRSelect的有效性。代码将公开。
原文摘要 · Abstract (English)
Deferred neural rendering (DNR) is an emerging computer graphics pipeline designed for high-fidelity rendering and robotic perception. However, DNR heavily relies on datasets composed of numerous ray-traced images and demands substantial computational resources. It remains under-explored how to reduce the reliance on high-quality ray-traced images while maintaining the rendering fidelity. In this paper, we propose DNRSelect, which integrates a reinforcement learning-based view selector and a 3D texture aggregator for deferred neural rendering. We first propose a novel view selector for deferred neural rendering based on reinforcement learning, which is trained on easily obtained rasterized images to identify the optimal views. By acquiring only a few ray-traced images for these selected views, the selector enables DNR to achieve high-quality rendering. To further enhance spatial awareness and geometric consistency in DNR, we introduce a 3D texture aggregator that fuses pyramid features from depth maps and normal maps with UV maps. Given that acquiring ray-traced images is more time-consuming than generating rasterized images, DNRSelect minimizes the need for ray-traced data by using only a few selected views while still achieving high-fidelity rendering results. We conduct detailed experiments and ablation studies on the NeRF-Synthetic dataset to demonstrate the effectiveness of DNRSelect. The code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。