用状态空间模型提升暗光与水下视频画质,效率更高
BVI-Mamba: Video Enhancement Using a Visual State-Space Model for Low-Light and Underwater Environments

- 用视觉状态空间模型替代传统卷积和注意力机制
- 在低光和水下场景中显著降低噪声并增强对比度
- 适合需要实时处理的视频增强应用
在暗光和水下环境中拍摄的视频常存在噪声、对比度低、色彩失衡和模糊等问题,不仅影响可视性,还损害检测等自动任务性能。传统后处理耗时,而基于AI的视频增强方法相比图像方法计算开销大。本文提出Visual Mamba框架,通过视觉状态空间(VSS)模型降低内存占用和计算时间。该框架包含两个模块:(i) 特征对齐模块,在特征空间中注册输入帧间的时空位移;(ii) 增强模块,采用类似UNet的结构,但所有卷积层替换为VSS块,实现去噪和亮度调整。实验表明,Visual Mamba在低光和水下视频增强任务中优于Transformer和卷积基模型。代码已公开于 https://github.com/russellllaputa/BVI-Mamba。
原文摘要 · Abstract (English)
Videos captured in low-light and underwater conditions often suffer from distortions such as noise, low contrast, color imbalance, and blur. These issues not only limit visibility but also degrade automatic tasks like detection. Post-processing is typically required but can be time-consuming. AI-based tools for video enhancement also demand significantly more computational resources compared to image-based methods. This paper introduces a novel framework, Visual Mamba, designed to reduce memory usage and computational time by leveraging the Visual State Space (VSS) model. The framework consists of two modules: (i) a feature alignment module, where spatio-temporal displacement between input frames is registered in the feature space, and (ii) an enhancement module, where noise removal and brightness adjustment are performed using a UNet-like architecture, with all convolutional layers replaced by VSS blocks. Experimental results show that the Visual Mamba technique outperforms Transformer and convolution-based models in both low-light and underwater video enhancement tasks. Code is available on line at https://github.com/russellllaputa/BVI-Mamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。