arXiv:2606.09882cs.CVcs.LG2026-06

构建3D道路基础设施数据集,助力城市数字孪生精准运维

WHU-Infra3D: A Full-stack Multi-modal Dataset and Benchmark for 3D Roadside Infrastructure Inventory

论文配图:WHU-Infra3D: A Full-stack Multi-modal Dataset and Benchmark for 3D Roadside Infrastructure Inventory
图 1 · 摘自论文原文
  • 融合全景图像与激光点云,实现2D-3D精准实例关联
  • 覆盖53.8公里,含181k属性与状态标注,支持缺陷诊断
  • 适合智能交通、城市运维与多模态感知研究者使用

城市数字孪生正从粗粒度视觉映射转向更精确、可操作的资产数字化。然而现有数据集多聚焦粗粒度视觉感知,缺乏严格多模态对齐及属性与状态诊断能力,难以支撑自动化维护。为此,我们提出WHU-Infra3D,一个大规模、多模态基准数据集,专用于道路基础设施盘点。覆盖三个城市共53.8公里,该数据集首次整合全景影像与激光点云,并实现严格的2D-3D实例关联与跨帧追踪。包含超过17.5万个多视角2D边界框及数千个3D基础设施实例,提供超18.1万条详细属性与状态标注(如锈蚀、遮挡),支持运行健康评估。我们在五个核心任务上建立全面基线:2D检测、2D跨视图匹配、3D地理标识、3D点云分割和属性识别。大量评估揭示显著的跨城市域差距及当前模型在长尾缺陷状态下的固有脆弱性,确立WHU-Infra3D作为推动可扩展AI驱动城市基础设施盘点与全生命周期管理的关键测试平台。数据集可在https://github.com/WHU-USI3DV/WHU-Infra3D获取。

原文摘要 · Abstract (English)

The paradigm of digital twin cities is shifting from coarse visual mapping toward more precise and actionable digitization of urban assets. However, existing datasets predominantly focus on coarse visual perception, lacking the strict multi-modal alignment and attribute and status diagnosis required for automated infrastructure maintenance. To bridge this gap, we introduce WHU-Infra3D, a large-scale, multi-modal benchmark dataset dedicated to roadside infrastructure inventory. Covering 53.8 km across three cities, WHU-Infra3D uniquely integrates panoramic imagery and LiDAR point clouds with rigorous 2D-3D instance association and cross-frame tracking. Comprising over 175k multi-view 2D bounding boxes alongside thousands of 3D infrastructure instances, the dataset provides over 181k detailed attribute and status annotations (e.g., rust, occlusion) to empower operational health assessment. We establish comprehensive baselines across five core tasks: 2D detection, 2D cross-view matching, 3D geo-identification, 3D point cloud segmentation, and attribute recognition. Extensive evaluations expose significant cross-city domain gaps and inherent vulnerabilities of current models on long-tailed defective statuses, establishing WHU-Infra3D as an essential testbed for advancing scalable, AI-driven urban infrastructure inventory and lifecycle management. The WHU-Infra3D dataset is available at https://github.com/WHU-USI3DV/WHU-Infra3D.

3D感知城市运维多模态数据数字孪生

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。