不依赖深度信息，纯视觉多摄像头3D追踪在真实场景中表现更优。
Geometry Beats Estimated Depth: RGB-Only Multi-Camera 3D Tracking under Sim2Real
- 用几何一致性替代单目深度估计，构建纯RGB的3D追踪流程。
- 几何优先方法3D HOTA达13.0（定位准确率51.6），伪激光雷达法仅0.12。
- 适用于缺乏深度数据、追求鲁棒性的工业级多视角追踪场景。
AI City Challenge 2026 Track 1 在大规模室内仓库中评估合成到真实（Sim2Real）环境下的多摄像头3D感知；训练与验证阶段提供深度信息，推理阶段为纯RGB输入。本文采用两条纯RGB路线进行对照实验，检验一个假设：跨视角几何一致性比单目深度精度更关键。第一条为几何优先流程：使用YOLO11x检测，通过单应性变换映射至世界坐标系，结合类别级3D尺寸先验，多相机融合，世界坐标跟踪，最后离线拼接轨迹。第二条为伪激光雷达方法：基于单目深度估计（D4RT、Metric3D~v2）反投影生成融合点云，输入V-DETR 3D检测器，模拟以往依赖点云的领先方法。结果差距显著：几何优先方法取得13.0的3D HOTA（51.6 LocA），而伪激光雷达方法崩溃至0.12（9.2 LocA）。我们发现失败根源在于单目深度跨视角不一致——尺度校正虽必要但不足以修复问题，且在预算范围内领域自适应微调无法解决。在几何流程中，唯一有效改进是离线轨迹拼接；其他尝试如SAHI检测、外观重识别、学习式提升、RT-DETR集成、测试时增强和领域随机化均未超越基线检测器。瓶颈互补：几何路线受限于检测质量（DetA），伪激光雷达受限于定位一致性（LocA）。论文开源完整可复现的纯RGB流程及消融实验。
原文摘要 · Abstract (English)
The AI City Challenge 2026 Track 1 evaluates multi-camera 3D perception in large indoor warehouses under a synthetic-to-real (Sim2Real) setting; depth is available only for training and validation, so inference is RGB-only. We use two RGB-only routes as a controlled test of one hypothesis: that cross-view geometric consistency, not monocular depth accuracy, governs performance under Sim2Real. The first is a geometry-first pipeline: YOLO11x detection, homography lifting to the world frame, class-level 3D size priors, multi-camera fusion, world-coordinate tracking, and offline tracklet stitching. The second is estimated-depth pseudo-LiDAR: monocular depth (D4RT, Metric3D~v2) back-projected into a fused point cloud and passed to a 3D detector (V-DETR), mirroring prior point-cloud winners that used depth. The gap is decisive: geometry-first reaches 13.0 3D HOTA (51.6 LocA), whereas pseudo-LiDAR collapses to 0.12 (9.2 LocA). We trace the collapse to cross-view inconsistency of monocular depth---scale correction is necessary but not sufficient---which domain-adaptation fine-tuning does not repair within budget. Within the geometry pipeline, offline stitching is the only intervention that helps; SAHI detection, appearance Re-ID, learned lifting, RT-DETR ensembling, test-time augmentation, and domain randomization all fail to beat the baseline detector. The bottlenecks are complementary: detection quality bounds the geometry route (DetA), localization consistency bounds pseudo-LiDAR (LocA). We release a complete, reproducible RGB-only pipeline and ablation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。