通过空间下采样提升等向网络效率,实现移动端高效去马赛克
Efficient Deep Demosaicing with Spatially Downsampled Isotropic Networks
- 提出在等向网络中引入显著空间下采样以降低计算开销
- 下采样版本JD3Net在多个任务上表现优于无下采样设计
- 适合移动设备部署的轻量级去马赛克与联合去噪场景
在数字成像中,图像去马赛克是从色彩滤波阵列(CFA)恢复RGB信息的关键步骤。深度学习常用于此任务。由于多数现代成像应用运行于移动平台,去马赛克的深度网络需轻量高效。等向网络(即残差嵌套网络)常被用于去马赛克及联合去马赛克-去噪(JDD)任务。然而,现有等向网络普遍避免空间下采样,导致计算成本过高,难以适用于移动端。本文提出相反观点:大幅空间下采样可提升等向网络的效率与性能。我们基于DeepMAD的数学架构设计方法,构建了有无下采样的全卷积网络,并发现下采样能提升实际性能。进一步测试表明,所提出的下采样变体JD3Net在多种去马赛克和JDD任务中表现优异。
原文摘要 · Abstract (English)
In digital imaging, image demosaicing is a crucial first step which recovers the RGB information from a color filter array (CFA). Oftentimes, deep learning is utilized to perform image demosaicing. Given that most modern digital imaging applications occur on mobile platforms, applying deep learning to demosaicing requires lightweight and efficient networks. Isotropic networks, also known as residual-in-residual networks, have been often employed for image demosaicing and joint-demosaicing-and-denoising (JDD). Most demosaicing isotropic networks avoid spatial downsampling entirely, and thus are often prohibitively expensive computationally for mobile applications. Contrary to previous isotropic network designs, this paper claims that spatial downsampling to a signficant degree can improve the efficiency and performance of isotropic networks. To validate this claim, we design simple fully convolutional networks with and without downsampling using a mathematical architecture design technique adapted from DeepMAD, and find that downsampling improves empirical performance. Additionally, empirical testing of the downsampled variant, JD3Net, of our fully convolutional networks reveals strong empirical performance on a variety of image demosaicing and JDD tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。