融合单目图像与稀疏点云，提升动态场景流估计精度与效率
SF3D-RGB: Scene Flow Estimation from Monocular Camera and Sparse LiDAR
- 双模态输入：2D图像与3D点云联合编码并融合特征
- 图匹配模块生成初始场景流，残差模块进一步优化结果
- 在真实数据集上优于单模态方法，参数更少且精度更高
场景流估计是计算机视觉中感知场景动态变化的关键任务。近年来，基于学习的方法在仅使用图像或仅使用激光雷达（LiDAR）数据的单模态方式上取得了显著进展。然而，这些方法多局限于单一模态。为此，本文提出一种端到端深度学习架构SF3D-RGB，利用单目图像和稀疏三维点云（如由LiDAR获取）进行场景流估计。模型首先分别对各模态信息编码为特征，再进行融合；随后，融合特征增强图匹配模块，以更鲁棒地计算映射矩阵，生成初始场景流；最后通过残差场景流模块进行细化。该方法兼顾精度与效率。实验表明，本方法在真实世界数据集上优于单模态方法，在保持更少参数的同时达到更高精度。
原文摘要 · Abstract (English)
Scene flow estimation is an extremely important task in computer vision to support the perception of dynamic changes in the scene. For robust scene flow, learning-based approaches have recently achieved impressive results using either image-based or LiDAR-based modalities. However, these methods have tended to focus on the use of a single modality. To tackle these problems, we present a deep learning architecture, SF3D-RGB, that enables sparse scene flow estimation using 2D monocular images and 3D point clouds (e.g., acquired by LiDAR) as inputs. Our architecture is an end-to-end model that first encodes information from each modality into features and fuses them together. Then, the fused features enhance a graph matching module for better and more robust mapping matrix computation to generate an initial scene flow. Finally, a residual scene flow module further refines the initial scene flow. Our model is designed to strike a balance between accuracy and efficiency. Furthermore, experiments show that our proposed method outperforms single-modality methods and achieves better scene flow accuracy on real-world datasets while using fewer parameters compared to other state-of-the-art methods with fusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。