为内存计算芯片设计视频去噪模型，提速77倍且画质接近顶尖水平。
CIM-NET: A Video Denoising Deep Neural Network Model Optimized for Computing-in-Memory Architectures
- 专为内存计算架构优化，用新型算子融合滑动与全连接操作。
- 步长8时矩阵乘法量降至原模型1/77，推理速度大幅提升。
- 适合边缘设备部署，兼顾实时性与能效，适合硬件协同设计研究者。
基于深度神经网络的视频去噪虽性能优异，但在边缘设备上部署仍面临实时性与能效的严峻挑战。计算内存（CIM）芯片通过在存储单元内集成计算，实现快速的矩阵-向量乘法（MVM）。然而，现有DNN模型多未考虑CIM架构约束，限制了其在推理阶段的加速潜力。为此，本文提出一种软硬件协同设计框架，包含两项创新：(1) 面向CIM的架构CIM-NET，针对大感受野操作和交叉阵列式MVM加速进行了优化；(2) 一种伪卷积算子CIM-CONV，用于在CIM-NET中融合基于滑动的处理与全连接变换，实现高质量特征提取与重建。该框架显著减少MVM操作次数，在保持竞争力性能的同时提升CIM芯片上的推理速度。实验表明，相较于传统轻量级模型FastDVDnet，CIM-NET在步长为8时将MVM操作减少至1/77，同时保持较高画质（PSNR 35.11 dB vs. 35.56 dB）。
原文摘要 · Abstract (English)
While deep neural network (DNN)-based video denoising has demonstrated significant performance, deploying state-of-the-art models on edge devices remains challenging due to stringent real-time and energy efficiency requirements. Computing-in-Memory (CIM) chips offer a promising solution by integrating computation within memory cells, enabling rapid matrix-vector multiplication (MVM). However, existing DNN models are often designed without considering CIM architectural constraints, thus limiting their acceleration potential during inference. To address this, we propose a hardware-algorithm co-design framework incorporating two innovations: (1) a CIM-Aware Architecture, CIM-NET, optimized for large receptive field operation and CIM's crossbar-based MVM acceleration; and (2) a pseudo-convolutional operator, CIM-CONV, used within CIM-NET to integrate slide-based processing with fully connected transformations for high-quality feature extraction and reconstruction. This framework significantly reduces the number of MVM operations, improving inference speed on CIM chips while maintaining competitive performance. Experimental results indicate that, compared to the conventional lightweight model FastDVDnet, CIM-NET substantially reduces MVM operations with a slight decrease in denoising performance. With a stride value of 8, CIM-NET reduces MVM operations to 1/77th of the original, while maintaining competitive PSNR (35.11 dB vs. 35.56 dB
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。