仅用一张真实图像,就能快速重建船舶3D模型,无需真实3D标注。
Synthetic-to-Real Domain Bridging for Single-View 3D Reconstruction of Ships for Maritime Monitoring
- 用合成数据训练3D高斯表示模型,实现单图快速重建。
- 在真实海事图像上重建误差低于15%,效果接近真实标注水平。
- 适合海事监控、智能航运系统实时部署,可交互查看船舶3D结构。
船舶三维重建是海事监控的关键技术,有助于提升可视化与决策能力。然而,现有方法多依赖多视角监督或高成本3D标注,难以实现实时部署。本文提出一种仅需单张图像的高效重建流程:完全基于合成数据训练,采用基于3D高斯的Splatter Image网络实现快速准确重建。模型先在合成的ShapeNet船体数据上微调,再通过自建的多样化3D船体数据集进一步优化,缩小合成与真实图像间的域差距。集成基于YOLOv8的分割模块和定制预处理,确保与重建网络兼容。后处理包括真实尺度校准、居中对齐及航迹匹配,结合AIS数据与单应性映射将结果放置于交互式网络地图。定量评估显示合成验证集上重建保真度优异,真实海事图像(ShipSG数据集)上的定性结果证实其在实际场景中的迁移潜力。最终系统可在无真实3D标注下实现真实船舶的交互式3D检测,为海事监控提供高效可扩展的解决方案。
原文摘要 · Abstract (English)
Three-dimensional (3D) reconstruction of ships is an important part of maritime monitoring, allowing improved visualization, inspection, and decision-making in real-world monitoring environments. However, most state-ofthe-art 3D reconstruction methods require multi-view supervision, annotated 3D ground truth, or are computationally intensive, making them impractical for real-time maritime deployment. In this work, we present an efficient pipeline for single-view 3D reconstruction of real ships by training entirely on synthetic data and requiring only a single view at inference. Our approach uses the Splatter Image network, which represents objects as sparse sets of 3D Gaussians for rapid and accurate reconstruction from single images. The model is first fine-tuned on synthetic ShapeNet vessels and further refined with a diverse custom dataset of 3D ships, bridging the domain gap between synthetic and real-world imagery. We integrate a state-of-the-art segmentation module based on YOLOv8 and custom preprocessing to ensure compatibility with the reconstruction network. Postprocessing steps include real-world scaling, centering, and orientation alignment, followed by georeferenced placement on an interactive web map using AIS metadata and homography-based mapping. Quantitative evaluation on synthetic validation data demonstrates strong reconstruction fidelity, while qualitative results on real maritime images from the ShipSG dataset confirm the potential for transfer to operational maritime settings. The final system provides interactive 3D inspection of real ships without requiring real-world 3D annotations. This pipeline provides an efficient, scalable solution for maritime monitoring and highlights a path toward real-time 3D ship visualization in practical applications. Interactive demo: https://dlr-mi.github.io/ship3d-demo/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。