用强化学习动态调分辨率，手机也能流畅看1080p逼真3D场景。
VistaFlow: Photorealistic Volumetric Reconstruction with Dynamic Resolution Management via Q-Learning
- 用Q-learning控制渲染分辨率，毫秒级调节画质与帧率。
- 在消费级硬件上实现1080p画面、超100帧每秒的实时渲染。
- 不依赖NeRF，轻量级PlenOctree结构适合手机等低配设备。
我们提出VistaFlow，一种可扩展的三维成像技术，能从一组二维照片重建出全交互式三维体数据。该模型通过可微分渲染系统合成新视角，并引入新控制器QuiQ，基于Q-learning实现毫秒级动态分辨率管理，以维持高帧率。VistaFlow原生运行于集成式CPU图形处理器，适用于移动设备和入门级硬件，仍可实现高性能渲染。其摒弃神经辐射场（NeRFs），采用PlenOctree数据结构，以极低硬件需求渲染复杂光照效果如反射与次表面散射。实验表明，VistaFlow在消费级硬件上以1080p分辨率实现超过100帧每秒的新视角合成，性能优于现有最优方法。通过适配不同设备能力，该模型有望显著提升跨硬件平台的逼真3D渲染效率与可及性。
原文摘要 · Abstract (English)
We introduce VistaFlow, a scalable three-dimensional imaging technique capable of reconstructing fully interactive 3D volumetric images from a set of 2D photographs. Our model synthesizes novel viewpoints through a differentiable rendering system capable of dynamic resolution management on photorealistic 3D scenes. We achieve this through the introduction of QuiQ, a novel intermediate video controller trained through Q-learning to maintain a consistently high framerate by adjusting render resolution with millisecond precision. Notably, VistaFlow runs natively on integrated CPU graphics, making it viable for mobile and entry-level devices while still delivering high-performance rendering. VistaFlow bypasses Neural Radiance Fields (NeRFs), using the PlenOctree data structure to render complex light interactions such as reflection and subsurface scattering with minimal hardware requirements. Our model is capable of outperforming state-of-the-art methods with novel view synthesis at a resolution of 1080p at over 100 frames per second on consumer hardware. By tailoring render quality to the capabilities of each device, VistaFlow has the potential to improve the efficiency and accessibility of photorealistic 3D scene rendering across a wide spectrum of hardware, from high-end workstations to inexpensive microcontrollers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。