arXiv:2410.20314cs.CVeess.IV2024-10被引 45

用小波与傅里叶结合的Mamba模型,提升暗光图像亮度和细节。

Wavelet-based Mamba with Fourier Adjustment for Low-light Image Enhancement

  • 融合小波低频亮度与傅里叶相位细节,分阶段增强图像。
  • 在SID、LOL数据集上均优于现有方法,速度更快、计算量更低。
  • 适合需要快速高质暗光增强的移动端或实时系统应用。

频率信息(如离散小波变换和快速傅里叶变换)已被广泛用于解决低光照图像增强(LLIE)问题。然而,现有基于频率的方法主要在图像的简单小波或傅里叶空间中操作,未能充分利用各空间中的全局与局部有效信息。我们发现,小波频率信息对全局亮度更敏感(因其低频分量),而傅里叶频率信息对局部细节更敏感(因其相位分量)。为通过最优融合空间通道信息与小波变换中的低频分量,实现优越的初始亮度增强,我们引入通道式Mamba,其可弥补CNN的长程依赖缺陷,且复杂度低于扩散模型与Transformer。本文提出一种新型小波-傅里叶调整的Mamba模型WalMaFa,由小波Mamba块(WMB)和快速傅里叶调整块(FFAB)构成,采用编码器-隐变量-解码器结构完成端到端转换。具体而言,WMB用于编码器与解码器以增强全局亮度,FFAB用于隐变量层以精细调整局部纹理并缓解歧义。大量实验表明,所提WalMaFa在计算资源更少、运行速度更快的前提下,达到当前最优性能。代码已开源:https://github.com/mcpaulgeorge/WalMaFa。

原文摘要 · Abstract (English)

Frequency information (e.g., Discrete Wavelet Transform and Fast Fourier Transform) has been widely applied to solve the issue of Low-Light Image Enhancement (LLIE). However, existing frequency-based models primarily operate in the simple wavelet or Fourier space of images, which lacks utilization of valid global and local information in each space. We found that wavelet frequency information is more sensitive to global brightness due to its low-frequency component while Fourier frequency information is more sensitive to local details due to its phase component. In order to achieve superior preliminary brightness enhancement by optimally integrating spatial channel information with low-frequency components in the wavelet transform, we introduce channel-wise Mamba, which compensates for the long-range dependencies of CNNs and has lower complexity compared to Diffusion and Transformer models. So in this work, we propose a novel Wavelet-based Mamba with Fourier Adjustment model called WalMaFa, consisting of a Wavelet-based Mamba Block (WMB) and a Fast Fourier Adjustment Block (FFAB). We employ an Encoder-Latent-Decoder structure to accomplish the end-to-end transformation. Specifically, WMB is adopted in the Encoder and Decoder to enhance global brightness while FFAB is adopted in the Latent to fine-tune local texture details and alleviate ambiguity. Extensive experiments demonstrate that our proposed WalMaFa achieves state-of-the-art performance with fewer computational resources and faster speed. Code is now available at: https://github.com/mcpaulgeorge/WalMaFa.

图像增强小波变换Mamba暗光处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。