用运动线索实现自动驾驶中未知物体的无类别分割。
Multiscale Video Transformers for Class Agnostic Segmentation in Autonomous Driving
- 多尺度视频变换器通过运动信息检测未知物体，无需已知类别训练。
- 在DAVIS'16、KITTI、Cityscapes上优于多尺度基线，且内存和推理效率高。
- 适合需要实时、鲁棒密集预测的安全关键机器人场景。
自动驾驶安全面临复杂挑战，需应对未知物体和突发驾驶场景。现有视频语义与全景分割依赖训练时见过的已知类别，难以处理新类别。近期基于大语言模型的视觉定位计算开销大，尤其对像素级输出不友好。本文提出一种端到端训练的高效视频变换器，实现无类别分割，无需光流。方法采用多阶段多尺度查询-记忆解码与尺度特定随机丢弃令牌机制，在保持高分辨率时空特征的同时提升效率。共享可学习的记忆模块避免特征压缩，保留多尺度细节。在DAVIS'16、KITTI和Cityscapes数据集上，方法持续超越多尺度基线，且在GPU内存占用和运行时间上表现优异，为安全关键机器人中的实时鲁棒密集预测提供了可行方向。
原文摘要 · Abstract (English)
Ensuring safety in autonomous driving is a complex challenge requiring handling unknown objects and unforeseen driving scenarios. We develop multiscale video transformers capable of detecting unknown objects using only motion cues. Video semantic and panoptic segmentation often relies on known classes seen during training, overlooking novel categories. Recent visual grounding with large language models is computationally expensive, especially for pixel-level output. We propose an efficient video transformer trained end-to-end for class-agnostic segmentation without optical flow. Our method uses multi-stage multiscale query-memory decoding and a scale-specific random drop-token to ensure efficiency and accuracy, maintaining detailed spatiotemporal features with a shared, learnable memory module. Unlike conventional decoders that compress features, our memory-centric design preserves high-resolution information at multiple scales. We evaluate on DAVIS'16, KITTI, and Cityscapes. Our method consistently outperforms multiscale baselines while being efficient in GPU memory and run-time, demonstrating a promising direction for real-time, robust dense prediction in safety-critical robotics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。