用事件相机数据高效计算深度，单目效果媲美立体方法。
DERD-Net: Learning Depth from Event-based Ray Densities
- 将事件数据转为射线密度图，结合3D卷积与循环结构预测深度。
- 单目场景深度误差接近立体方法，立体下误差降低超42%。
- 适合高动态范围、高速场景的实时深度估计，适合机器人导航。
事件相机能在高速和宽光照条件下捕捉无模糊的3D边缘，为多视角立体深度估计与同步定位建图（SLAM）提供了新路径。但传统针对普通相机设计的深度学习框架难以处理事件数据的异步流式特性。本文提出一种可扩展、灵活且适应性强的像素级深度估计框架，适用于单目与立体设置。通过已知相机位姿将事件反投影至空间，生成表征射线空间密度的视差空间图像（DSI）。神经网络对DSI的局部子区域进行处理，融合3D卷积与循环结构以识别深度预测的有效模式。局部处理支持快速推理与完全并行化，模型复杂度与内存开销恒定，不受相机分辨率影响。在标准基准（MVSEC与DSEC数据集）上的实验表明：（i）仅使用单目数据，性能可媲美现有立体方法；（ii）应用于立体数据时，显著优于所有当前最先进方法，平均绝对误差降低至少42%；（iii）深度完整性提升超过3倍，同时中位绝对误差仍降低至少30%。该框架在事件数据处理与深度估计方面表现卓越，具备成为事件相机深度估计与SLAM标准方法的巨大潜力。
原文摘要 · Abstract (English)
Event cameras offer a promising avenue for multi-view stereo depth estimation and Simultaneous Localization And Mapping (SLAM) due to their ability to detect blur-free 3D edges at high-speed and over broad illumination conditions. However, traditional deep learning frameworks designed for conventional cameras struggle with the asynchronous, stream-like nature of event data, as their architectures are optimized for discrete, image-like inputs. We propose a scalable, flexible and adaptable framework for pixel-wise depth estimation with event cameras in both monocular and stereo setups. The 3D scene structure is encoded into disparity space images (DSIs), representing spatial densities of rays obtained by back-projecting events into space via known camera poses. Our neural network processes local subregions of the DSIs combining 3D convolutions and a recurrent structure to recognize valuable patterns for depth prediction. Local processing enables fast inference with full parallelization and ensures constant ultra-low model complexity and memory costs, regardless of camera resolution. Experiments on standard benchmarks (MVSEC and DSEC datasets) demonstrate unprecedented effectiveness: (i) using purely monocular data, our method achieves comparable results to existing stereo methods; (ii) when applied to stereo data, it strongly outperforms all state-of-the-art (SOTA) approaches, reducing the mean absolute error by at least 42%; (iii) our method also allows for increases in depth completeness by more than 3-fold while still yielding a reduction in median absolute error of at least 30%. Given its remarkable performance and effective processing of event-data, our framework holds strong potential to become a standard approach for using deep learning for event-based depth estimation and SLAM. Project page: https://github.com/tub-rip/DERD-Net
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。