用脉冲神经网络思想+状态空间模型,让去雾模型在真实照片上表现更好。
HazeSpikeMamba: Coupling Spiking-Inspired and State-Space Features for Self-Supervised Real-World Dehazing
- 结合脉冲启发局部路径与状态空间全局路径,多尺度建模雾霾特征。
- 无需真实清晰图像,在未标注数据上自适应后BRISQUE下降至27.72,NIMA升至4.87。
- 适合追求真实场景去雾效果、资源有限的工业应用或移动端部署。
去雾网络通常在合成的有雾-无雾配对数据上训练,但在真实照片上性能下降明显。合成雾霾虽基于大气散射模型,但难以涵盖真实雾霾的多样性,且真实配对数据稀缺。本文提出HazeSpikeMamba,一个紧凑的去雾框架,融合脉冲启发的局部路径与注意力状态空间全局路径,构建多尺度U-Net结构。局部路径采用新型脉冲发射机制TPCNNSpike,受脉冲耦合神经网络(PCNN)启发,通过高斯加权邻域前次放电状态并行更新所有神经元,替代分组方向扫描。全局路径改进MambaIRv2的注意力状态空间模块,保留语义提示和序列重排,移除窗口自注意力分支,以线性复杂度建模长程依赖。为实现目标域自适应,使用在配对NH-HAZE数据上预训练的冻结退化网络,从去雾预测中重建雾霾。仅更新最终恢复层,无需无雾标签。共享检查点在每个完整无标签目标集上适应一次,评估为数据集级、归纳式而非零样本或逐图优化。前向网络含202万活跃参数,13.27G名义乘加操作(thop测量,输入256x256)。该自适应方法在RTTS、URHI和HSTS上持续提升BRISQUE与NIMA表现;在RTTS上,BRISQUE从30.13降至27.72,NIMA从4.13升至4.87。在此归纳协议下,该模型在URHI和HSTS上的BRISQUE与NIMA均优于对比方法。
原文摘要 · Abstract (English)
Dehazing networks are commonly trained on synthetic hazy-clear pairs, but their performance often drops on real photographs. Synthetic haze generated using the atmospheric scattering model does not fully capture the variability of real haze, and paired real hazy-clear images are scarce. In this work, we propose HazeSpikeMamba, a compact dehazing framework that combines a spiking-inspired local path and an attentive state-space global path in a multi-scale U-Net. The local path uses TPCNNSpike, a new spike-emission scheme inspired by the neighborhood coupling of Pulse-Coupled Neural Network (PCNN). Unlike grouped directional scanning, TPCNNSpike updates all neurons in parallel using the previous firing states of their Gaussian-weighted neighborhoods. The global path adapts the Attentive State-Space Module of MambaIRv2, retaining semantic prompting and sequence reordering while removing the window self-attention branch. Its state-space processing models long-range dependencies with complexity linear in sequence length. For target-domain adaptation, a frozen degradation network, pretrained on paired NH-HAZE data, re-synthesizes haze from the dehazed prediction. The reconstruction error updates only the final restoration layers of HazeSpikeMamba without haze-free labels during adaptation. A shared checkpoint is adapted once on each complete unlabeled target set, making the evaluation dataset-level and transductive rather than zero-shot or per-image optimization. The forward network contains 2.02M active parameters and requires 13.27G nominal MACs (measured with thop at 256x256 input). This adaptation consistently improves BRISQUE and NIMA on RTTS, URHI, and HSTS. On RTTS, BRISQUE decreases from 30.13 to 27.72 and NIMA increases from 4.13 to 4.87. Under this transductive protocol, the adapted model also achieves the best BRISQUE and NIMA on URHI and HSTS among the compared methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。