用互补视觉传感器的时空差异信息，实现极端运动下的清晰图像恢复。
Spatio-Temporal Difference Guided Motion Deblurring with the Complementary Vision Sensor
- 通过递归多分支网络融合空间与时间差异信号
- 在合成与真实场景中均超越现有方法，还原细节更优
- 适合动态剧烈、传统方法失效的成像场景
运动模糊发生在快速场景变化期间，导致曝光期内的丰富运动信息坍缩为单个RGB帧。仅依赖RGB的去模糊任务因缺乏结构或时间线索而高度病态，在极端运动下常失败。受人眼视觉系统启发，脑启发式视觉传感器引入高密度时序信息以缓解该问题。然而，事件相机在快速运动下仍存在事件率饱和问题，且事件模态混杂边缘与运动信息，限制其效果。近期突破性成果——互补视觉传感器（CVS，Tianmouc）可在单次RGB曝光内同步捕获高帧率、多比特的空间差异（SD，编码结构边缘）和时间差异（TD，编码运动线索）数据，为极端动态场景下的RGB去模糊提供新方案。为充分利用这些互补模态，本文提出时空差异引导去模糊网络（STGDNet），采用递归多分支架构，迭代编码并融合SD与TD序列，恢复模糊RGB输入中丢失的结构与色彩细节。实验表明，该方法在合成CVS数据集及真实世界测试中均优于现有基于RGB或事件的方法，并在超过100种极端真实场景中表现出强泛化能力。
原文摘要 · Abstract (English)
Motion blur arises when rapid scene changes occur during the exposure period, collapsing rich intra-exposure motion into a single RGB frame. Without explicit structural or temporal cues, RGB-only deblurring is highly ill-posed and often fails under extreme motion. Inspired by the human visual system, brain-inspired vision sensors introduce temporally dense information to alleviate this problem. However, event cameras still suffer from event rate saturation under rapid motion, while the event modality entangles edge features and motion cues, which limits their effectiveness. As a recent breakthrough, the complementary vision sensor (CVS), Tianmouc, captures synchronized RGB frames together with high-frame-rate, multi-bit spatial difference (SD, encoding structural edges) and temporal difference (TD, encoding motion cues) data within a single RGB exposure, offering a promising solution for RGB deblurring under extreme dynamic scenes. To fully leverage these complementary modalities, we propose Spatio-Temporal Difference Guided Deblur Net (STGDNet), which adopts a recurrent multi-branch architecture that iteratively encodes and fuses SD and TD sequences to restore structure and color details lost in blurry RGB inputs. Our method outperforms current RGB or event-based approaches in both synthetic CVS dataset and real-world evaluations. Moreover, STGDNet exhibits strong generalization capability across over 100 extreme real-world scenarios. Project page: https://tmcDeblur.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。