arXiv:2501.15808cs.CV2025-01ICCV被引 6

受人眼注意力启发,用混合神经网络提升事件相机运动去模糊效果

ClearSight: Human Vision-Inspired Solutions for Event-Based Motion Deblurring

  • 结合脉冲神经网络与人工神经网络,分治运动与颜色信息处理
  • 动态调节神经元配置,聚焦模糊区域,适应不同模糊程度
  • 无监督生成模糊掩码,精准引导跨模态特征融合,适合真实场景应用

运动去模糊旨在解决由相机或场景移动引起的图像模糊问题。事件相机通过异步事件流编码运动信息。为高效利用事件流的时间信息,本文采用脉冲神经网络(SNNs)提取运动特征,人工神经网络(ANNs)处理颜色信息。由于事件数据分布不均且存在冗余,现有跨模态特征融合方法存在局限。受人类视觉系统中视觉注意机制启发,提出生物启发的双驱动混合网络(BDHNet)。具体地,神经元配置模块(NCM)根据跨模态特征动态调整神经元配置,聚焦模糊区域,自适应不同模糊场景。此外,模糊区域注意力模块(RBAM)以无监督方式生成模糊掩码,有效从事件特征中提取运动线索,指导更精确的跨模态特征融合。在合成与真实数据集上的主观与客观评估均表明,该方法优于当前最先进方法。

原文摘要 · Abstract (English)

Motion deblurring addresses the challenge of image blur caused by camera or scene movement. Event cameras provide motion information that is encoded in the asynchronous event streams. To efficiently leverage the temporal information of event streams, we employ Spiking Neural Networks (SNNs) for motion feature extraction and Artificial Neural Networks (ANNs) for color information processing. Due to the non-uniform distribution and inherent redundancy of event data, existing cross-modal feature fusion methods exhibit certain limitations. Inspired by the visual attention mechanism in the human visual system, this study introduces a bioinspired dual-drive hybrid network (BDHNet). Specifically, the Neuron Configurator Module (NCM) is designed to dynamically adjusts neuron configurations based on cross-modal features, thereby focusing the spikes in blurry regions and adapting to varying blurry scenarios dynamically. Additionally, the Region of Blurry Attention Module (RBAM) is introduced to generate a blurry mask in an unsupervised manner, effectively extracting motion clues from the event features and guiding more accurate cross-modal feature fusion. Extensive subjective and objective evaluations demonstrate that our method outperforms current state-of-the-art methods on both synthetic and real-world datasets.

事件相机运动去模糊脉冲网络注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。