用神经形态事件流修复视频中的局部光照伪影
DeLux: Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data
- 利用神经形态事件流作为结构先验，指导RGB视频中光照伪影的定位与修复
- 在真实车载视频上实现伪影严重度降低88%，平均MS-SSIM超0.99
- 适合关注多模态视觉修复、自动驾驶感知的开发者和研究者
传统RGB相机在光照条件下易出现耀斑、眩光、闪烁和过曝等伪影，导致信息不可逆丢失，需通过计算手段恢复。现有方法多孤立处理各类问题，难以完全还原被复杂空间离散退化遮蔽的结构细节。本文提出一种新型跨模态恢复范式，并构建DeLux模块化原型系统，利用神经形态事件流作为结构先验，引导对RGB视频中光照伪影的精准检测与修补。在合成基准和真实车载视频上的验证表明，DeLux能有效抑制局部伪影并恢复受损区域。相比现有仅基于RGB的基线方法及事件引导的HDR模型，该方法在所有伪影类型上平均MS-SSIM超过0.99，并在真实车载数据中实现最高88%的伪影严重度降低。合成伪影生成工具与精选的真实世界评估数据集已公开，以促进跨模态恢复研究。
原文摘要 · Abstract (English)
Conventional RGB cameras suffer from lighting artifacts such as flare, glare, flicker, and overexposure, leading to irrecoverable information loss that necessitates computational restoration. However, existing approaches treat these problems in isolation, failing to recover structural details completely obscured by complex spatially discrete image degradations. In this paper, we propose a novel cross-modal restoration paradigm and present DeLux, a modular proof-of-concept pipeline that leverages neuromorphic event streams as a structural prior to guide the targeted detection and inpainting of lighting artifacts in RGB video. Validation on synthetic benchmarks and real-world automotive footage demonstrates that DeLux effectively suppresses local artifacts and restores affected regions. The proposed approach outperforms existing RGB-only baselines and event-guided HDR models, achieving an average MS-SSIM of over 0.99 across all artifact types and demonstrating up to an 88% reduction in artifact severity in real-world automotive footage. The synthetic artifact generation tools and curated real-world evaluation datasets are made publicly available to foster future research on cross-modal restoration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。