通过层缓存技术加速实时渲染中的神经网络推理。
ReFrame: Layer Caching for Accelerated Inference in Real-Time Rendering
- 利用前帧中间特征缓存减少重复计算。
- 平均提速1.4倍，画质损失可忽略。
- 适用于去噪、超采样等常见渲染任务。
图形渲染应用越来越多地使用神经网络完成去噪、超采样和帧外推等任务，以在保持帧率的同时提升图像质量。这些任务固有的时间连贯性为复用前一帧的中间结果、避免冗余计算提供了机会。已有研究证明，在扩散模型中缓存中间特征可有效降低延迟。本文将该思想扩展至实时渲染，提出ReFrame，探索多种缓存策略以优化渲染负载下质量和性能的权衡。ReFrame适用于渲染流水线中常见的编码器-解码器结构网络。实验结果表明，在三项实时渲染任务中，平均实现1.4倍加速，且画质损失微不足道。代码已公开：https://ubc-aamodt-group.github.io/reframe-layer-caching/
原文摘要 · Abstract (English)
Graphics rendering applications increasingly leverage neural networks in tasks such as denoising, supersampling, and frame extrapolation to improve image quality while maintaining frame rates. The temporal coherence inherent in these tasks presents an opportunity to reuse intermediate results from previous frames and avoid redundant computations. Recent work has shown that caching intermediate features to be reused in subsequent inferences is an effective method to reduce latency in diffusion models. We extend this idea to real-time rendering and present ReFrame, which explores different caching policies to optimize trade-offs between quality and performance in rendering workloads. ReFrame can be applied to a variety of encoder-decoder style networks commonly found in rendering pipelines. Experimental results show that we achieve 1.4x speedup on average with negligible quality loss in three real-time rendering tasks. Code available: https://ubc-aamodt-group.github.io/reframe-layer-caching/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。