首个支持多种神经渲染的实时边缘加速器,让设备本地运行3D渲染更高效。
Uni-Render: A Unified Accelerator for Real-Time Rendering Across Diverse Neural Renderers
- 设计可重构硬件,动态适配不同渲染工作负载
- 在真实场景和合成场景中均实现实时渲染性能
- 适合开发下一代神经图形应用的工程师和研究者
神经渲染技术及其支撑设备的进展推动了沉浸式3D体验的发展,显著改变了人机交互方式。然而,实现沉浸式交互所需的实时渲染速度仍受制于(1)缺乏适用于多种应用场景的通用算法方案,以及(2)现有设备或加速器仅针对特定渲染管线优化。为此,我们开发了一款统一神经渲染加速器,可支持多种典型神经渲染管道,在保持效率与兼容性的前提下,实现在不同应用中的实时、本地渲染。该加速器的设计基于一个关键洞察:尽管神经渲染管道多样且算法持续演进,但其普遍共享常见算子,执行相似工作负载。基于此,我们提出一种可重构硬件架构,能动态调整数据流以匹配不同应用的渲染指标需求,有效支持典型及最新混合渲染管线。在合成与真实场景上的基准测试与消融实验验证了该加速器的有效性。所提方案是首个可在边缘设备上跨多种代表性渲染管道实现实时神经渲染的解决方案,有望为下一代神经图形应用铺平道路。
原文摘要 · Abstract (English)
Recent advancements in neural rendering technologies and their supporting devices have paved the way for immersive 3D experiences, significantly transforming human interaction with intelligent devices across diverse applications. However, achieving the desired real-time rendering speeds for immersive interactions is still hindered by (1) the lack of a universal algorithmic solution for different application scenarios and (2) the dedication of existing devices or accelerators to merely specific rendering pipelines. To overcome this challenge, we have developed a unified neural rendering accelerator that caters to a wide array of typical neural rendering pipelines, enabling real-time and on-device rendering across different applications while maintaining both efficiency and compatibility. Our accelerator design is based on the insight that, although neural rendering pipelines vary and their algorithm designs are continually evolving, they typically share common operators, predominantly executing similar workloads. Building on this insight, we propose a reconfigurable hardware architecture that can dynamically adjust dataflow to align with specific rendering metric requirements for diverse applications, effectively supporting both typical and the latest hybrid rendering pipelines. Benchmarking experiments and ablation studies on both synthetic and real-world scenes demonstrate the effectiveness of the proposed accelerator. The proposed unified accelerator stands out as the first solution capable of achieving real-time neural rendering across varied representative pipelines on edge devices, potentially paving the way for the next generation of neural graphics applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。