arXiv:2605.16911cs.CV2026-05

用几何约束提升3D占据预测精度,兼顾效率与准确性。

VGGT-Occ: Geometry-Grounded and Density-Aware Gated Fusion for 3D Occupancy Prediction

论文配图:VGGT-Occ: Geometry-Grounded and Density-Aware Gated Fusion for 3D Occupancy Prediction
图 1 · 摘自论文原文
  • 全程嵌入几何令牌,通过投影感知注意力注入物理约束
  • SurroundOcc-nuScenes上达33.64% IoU,仅需约41M参数
  • 适合追求高精度3D场景理解的自动驾驶研究者

3D语义占据预测需要精确的2D到3D特征提升,但现有方法将相机几何仅限于初始投影。后续的偏移学习、注意力加权和跨视角聚合仍忽略物理约束。我们提出VGGT-Occ,将几何令牌贯穿整个流程。引入投影感知可变形注意力(PA-DA),将3D偏移投影回图像平面,并利用投影雅可比作为加性偏差,抑制不可靠观测。特征通过视图质量语义门融合以保证跨视角一致性。为优化效率与性能,采用分阶段粗到细解码器,通过门控融合按信息密度分配计算量,显著降低解码器成本。大量实验表明该方法有效且准确:在SurroundOcc-nuScenes上,$T=1$时达33.00% IoU和21.08% mIoU,$T=2$时达33.64% IoU和21.43% mIoU,占用头仅约4100万可训练参数。代码将公开。

原文摘要 · Abstract (English)

3D semantic occupancy prediction requires accurate 2D-to-3D feature lifting, yet current methods restrict camera geometry to initial projections. Subsequent operations like offset learning, attention weighting, and cross-camera aggregation remain geometry-agnostic, ignoring essential physical constraints. We propose VGGT-Occ, a framework that embeds geometric tokens throughout the entire pipeline. We introduce Projection-Aware Deformable Attention (PA-DA) to inject geometry into all attention stages. PA-DA projects 3D offsets back to image planes and leverages the projection Jacobian as an additive bias to suppress unreliable observations. Features are then integrated through a view-quality semantic gate for cross-view consistency. To optimize both efficiency and performance, we employ a sequential coarse-to-fine decoder with gated fusion, where low-resolution features are refined into higher resolutions, allocating computation by information density while substantially reducing decoder cost. Extensive evaluations demonstrate the effectiveness and accuracy of our approach. On SurroundOcc-nuScenes, VGGT-Occ achieves 33.00\% IoU and 21.08\% mIoU ($T{=}1$), and 33.64\% IoU and 21.43\% mIoU with $T{=}2$ inference, outperforming existing methods, with only ${\sim}41$M trainable parameters in the occupancy head. Code will be released publicly.

3D占据几何约束多视角融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。