提出高效在线深度感知视频全景分割框架，支持自动驾驶实时环境理解。
DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving
- 用显式场景离散化机制，通过查询统一建模语义与几何信息
- 单次推理完成深度估计，比现有方法速度提升30%以上
- 适合需要实时感知的自动驾驶系统，尤其关注动态目标跟踪
安全的自动驾驶需对动态环境进行整体理解，要求同时估计度量深度、语义分割和实例轨迹。尽管深度感知视频全景分割（DVPS）统一了这些任务，但现有方法通常依赖计算成本高的多阶段流水线或离线追踪，难以满足实时决策需求。为此，我们提出DVPSFormer，一种面向高效4D场景理解的统一在线架构。核心是显式场景离散化（ESD），利用分割查询表示前景与背景区域，使离散到连续（D2C）深度头能单次推理解码度量深度，紧密耦合语义与几何学习并显著降低延迟。此外，提出在线多数投票（OMV）机制，利用时序一致性在实例追踪中优化分类。DVPSFormer在Cityscapes-DVPS和SemKITTI-DVPS基准上达到新最优性能，为在线机器人感知提供简洁解决方案。代码与模型已公开于https://royyang0714.github.io/DVPSFormer。
原文摘要 · Abstract (English)
Safe autonomous navigation requires a holistic understanding of dynamic environments, necessitating the simultaneous estimation of metric depth, semantic segmentation, and instance trajectories. While depth-aware video panoptic segmentation (DVPS) unifies these tasks, existing approaches often rely on computationally expensive, multi-stage pipelines or offline tracking, rendering them unsuitable for real-time decision-making. To address this, we propose DVPSFormer, a unified online architecture designed for efficient 4D scene understanding. Central to our approach is explicit scene discretization (ESD), a novel mechanism that leverages segmentation queries to represent foreground and background regions, enabling a discrete-to-continuous (D2C) depth head to decode metric depth in a single pass. This tightly couples semantic and geometric learning while significantly reducing latency. Furthermore, we propose an online majority voting (OMV) mechanism that exploits temporal consistency to refine classification during instance tracking. DVPSFormer establishes a new state-of-the-art on the Cityscapes-DVPS and SemKITTI-DVPS benchmarks, offering a streamlined solution for online robotic perception. Code and models are available at https://royyang0714.github.io/DVPSFormer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。