提出RED模型,解决事件相机运动模糊去模糊中的信号缺失问题。
RED: Robust Event-Guided Motion Deblurring with Modality-Specific Disentanglement
- 分离图像语义、事件运动与跨模态信息,针对性提取有效运动线索。
- 在真实数据集上实现比现有方法更高的精度和鲁棒性,峰值性能提升显著。
- 适合做视频去模糊、自动驾驶感知等对动态画面清晰度要求高的场景。
事件引导的运动模糊去模糊利用事件相机提供的高时间分辨率运动信息重建清晰图像。然而,在真实拍摄中,阈值触发导致的事件漏报会造成运动线索缺失或断裂,现有方法因两大缺陷性能下降:一是假设事件密集且稳定,二是不区分模态地提取融合,无法分离有用运动信号与干扰事件,导致跨模态表示被污染。本文首先提出一种面向鲁棒性的扰动策略(RPS),模拟动态视觉传感器的不同触发阈值,使模型暴露于多种漏报模式,从而提升对未知条件的适应能力。在此基础上,提出鲁棒事件引导去模糊网络RED,遵循‘先解耦,再选择性融合’原则。具体地,模态特定表示机制将输入解耦为图像语义、事件运动与跨模态三类表征,分别捕捉外观、运动及互补交互信息。基于可靠解耦特征,选择性融合模态以增强模糊图像中的运动敏感区域,并用语义上下文补充低报事件。在合成与真实世界数据集上的大量实验表明,RED在准确率与鲁棒性方面均达到当前最优水平。
原文摘要 · Abstract (English)
Event-guided motion deblurring reconstructs sharp images using the high-temporal-resolution motion cues from event cameras. However, in real capture, thresholding-induced event under-reporting causes missing and fragmented motion cues, under which existing methods often degrade in performance due to two limitations: i) assumptions of dense and stable events, and ii) modality-indiscriminate extraction and fusion that fail to separate useful motion cues from disrupted events, allowing them to contaminate cross-modal representations. In this paper, we first introduce a Robustness-Oriented Perturbation Strategy (RPS) that mimics various trigger thresholds of dynamic vision sensors, exposing our model to diverse under-reporting patterns and thereby improving robustness under unknown conditions. Built upon this setting, we propose RED, a Robust Event-guided Deblurring network, following the principle of disentangle first and then fuse selectively. Specifically, the Modality-specific Representation Mechanism disentangles the inputs into image-semantic, event-motion, and cross-modal representations, capturing appearance, motion, and complementary interactions, respectively. With the reliable disentangled features, we selectively fuse modalities to enhance motion-sensitive areas in blurry images and enrich under-reported events with semantic context. Extensive experiments on synthetic and real-world datasets demonstrate RED consistently achieves state-of-the-art performance in terms of both accuracy and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。