用手机激光雷达的深度信息提升模糊图像修复效果。
Deep Lidar-guided Image Deblurring
- 设计通用适配器，将深度数据融入现有去模糊模型。
- 在真实手机激光雷达数据集上，去模糊性能显著提升。
- 仅需少量额外数据，即可让预训练模型支持深度输入。
便携式激光雷达设备（如智能手机中的）的普及，为新型计算成像技术提供了可能。作为主动传感设备，激光雷达可为被动光学传感器提供互补数据，尤其在低光条件下运动模糊问题突出时。本文研究了手机激光雷达提供的深度信息对图像去模糊任务的有效性，并提出一种通用适配结构，可将任意先进神经去模糊模型转化为深度感知模型。该结构高效预处理深度信息，以调制图像特征。此外，采用持续学习策略对预训练编码器-解码器模型进行微调，使其以最小额外数据需求，将深度信息作为新增输入。实验表明，利用真实世界手机激光雷达采集的深度数据，可显著提升去模糊算法性能。
原文摘要 · Abstract (English)
The rise of portable Lidar instruments, including their adoption in smartphones, opens the door to novel computational imaging techniques. Being an active sensing instrument, Lidar can provide complementary data to passive optical sensors, particularly in situations like low-light imaging where motion blur can affect photos. In this paper, we study if the depth information provided by mobile Lidar sensors is useful for the task of image deblurring and how to integrate it with a general approach that transforms any state-of-the-art neural deblurring model into a depth-aware one. To achieve this, we developed a universal adapter structure that efficiently preprocesses the depth information to modulate image features with depth features. Additionally, we applied a continual learning strategy to pretrained encoder-decoder models, enabling them to incorporate depth information as an additional input with minimal extra data requirements. We demonstrate that utilizing true depth information can significantly boost the effectiveness of deblurring algorithms, as validated on a dataset with real-world depth data captured by a smartphone Lidar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。