通过混合现实头显实时看到点云重建,快速发现室内建模盲区。
3D Reconstruction by Looking: Instantaneous Blind Spot Detector for Indoor SLAM through Mixed Reality
- 用混合现实头显将点云以全息影像叠加在真实场景上,实时可视化配准结果。
- 盲区检测F1分数达75.76%,在五个区域的图像相似性最高达0.5619。
- 适合建筑信息模型等需要高精度3D重建的现场作业人员使用。
室内SLAM常因场景漂移、双墙伪影和盲区问题导致重建质量下降,尤其在传感器附近物体密集的空间中更为显著。为改善此状况,我们开发了LiMRSF(LiDAR-MR-RGB传感器融合)系统,使用户通过混合现实头显实时观察点云配准效果。该系统将点云网格以全息形式呈现,并与实时场景无缝对齐,自动标注重叠误差。全息数据通过TCP服务器传输至头显,经世界坐标校准后实现物理位置精准匹配。用户可即时发现盲区与错误并现场修正。实验显示,该盲区检测器在五段简化网格模型中达到最高结构相似性(SSIM 0.5619)、峰值信噪比(PSNR 14.1004)及最低均方误差(MSE 0.0389),F1得分75.76%。该方法可生成高质量3D数据集,适用于建筑信息模型(BIM)等场景。
原文摘要 · Abstract (English)
Indoor SLAM often suffers from issues such as scene drifting, double walls, and blind spots, particularly in confined spaces with objects close to the sensors (e.g. LiDAR and cameras) in reconstruction tasks. Real-time visualization of point cloud registration during data collection may help mitigate these issues, but a significant limitation remains in the inability to in-depth compare the scanned data with actual physical environments. These challenges obstruct the quality of reconstruction products, frequently necessitating revisit and rescan efforts. For this regard, we developed the LiMRSF (LiDAR-MR-RGB Sensor Fusion) system, allowing users to perceive the in-situ point cloud registration by looking through a Mixed-Reality (MR) headset. This tailored framework visualizes point cloud meshes as holograms, seamlessly matching with the real-time scene on see-through glasses, and automatically highlights errors detected while they overlap. Such holographic elements are transmitted via a TCP server to an MR headset, where it is calibrated to align with the world coordinate, the physical location. This allows users to view the localized reconstruction product instantaneously, enabling them to quickly identify blind spots and errors, and take prompt action on-site. Our blind spot detector achieves an error detection precision with an F1 Score of 75.76% with acceptably high fidelity of monitoring through the LiMRSF system (highest SSIM of 0.5619, PSNR of 14.1004, and lowest MSE of 0.0389 in the five different sections of the simplified mesh model which users visualize through the LiMRSF device see-through glasses). This method ensures the creation of detailed, high-quality datasets for 3D models, with potential applications in Building Information Modeling (BIM) but not limited.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。