用FPGA加速视频去噪,效率比之前高5倍,还能无缝接入现有工作流。
ReTiDe: Real-Time Denoising for Energy-Efficient Motion Picture Processing with FPGAs
- 轻量卷积模型量化为INT8,在FPGA上部署推理
- 每秒处理37.71亿次操作,能效提升5.29倍
- 兼容NUKE等工具,去噪质量几乎无损
去噪是现代视频流水线的核心操作。在编码器中,环内滤波器可抑制传感器噪声和量化伪影,提升码率-失真性能;在电影后期制作中,去噪器用于修复、颗粒管理与画面清理。然而,当前先进的深度去噪模型计算开销大,大规模部署通常依赖GPU,导致实时高分辨率流的功耗与成本过高。本文提出ReTiDe,一种基于数据中心现场可编程门阵列(FPGA)的实时去噪系统。采用紧凑型卷积模型,经后训练量化与量化感知微调转为INT8,并编译至AMD深度学习处理器单元(DPU)型FPGA。通过客户端-服务器架构,将计算任务从主机CPU/GPU卸载至网络化FPGA服务,同时保持对现有工作流(如NUKE)的兼容性。在典型基准测试中,ReTiDe实现37.71×10⁹次每秒(GOPS)吞吐量,相比先前FPGA去噪加速器能效提升5.29倍,峰值信噪比(PSNR)/结构相似性指数(SSIM)仅轻微下降。结果表明,专用加速器可为编码管道与后期制作提供高效、可扩展的去噪方案,在不牺牲画质或工作流兼容性的前提下显著降低每帧能耗。代码已开源:https://github.com/RCSL-TCD/ReTiDe。
原文摘要 · Abstract (English)
Denoising is a core operation in modern video pipelines. In codecs, in-loop filters suppress sensor noise and quantisation artefacts to improve rate-distortion performance; in cinema post-production, denoisers are used for restoration, grain management, and plate clean-up. However, state-of-the-art deep denoisers are computationally intensive and, at scale, are typically deployed on GPUs, incurring high power and cost for real-time, high-resolution streams. This paper presents Real-Time Denoise (ReTiDe), a hardware-accelerated denoising system that serves inference on data-centre Field Programmable Gate Arrays (FPGAs). A compact convolutional model is quantised (post-training quantisation plus quantisation-aware fine-tuning) to INT8 and compiled for AMD Deep Learning Processor Unit (DPU)-based FPGAs. A client-server integration offloads computation from the host CPU/GPU to a networked FPGA service, while remaining callable from existing workflows, e.g., NUKE, without disrupting artist tooling. On representative benchmarks, ReTiDe delivers 37.71$\times$ Giga Operations Per Second (GOPS) throughput and 5.29$\times$ higher energy efficiency than prior FPGA denoising accelerators, with negligible degradation in Peak Signal-to-Noise Ratio (PSNR)/Structural Similarity Index (SSIM). These results indicate that specialised accelerators can provide practical, scalable denoising for both encoding pipelines and post-production, reducing energy per frame without sacrificing quality or workflow compatibility. Code is available at https://github.com/RCSL-TCD/ReTiDe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。