arXiv:2409.00618cs.CV2024-09TPAMI被引 2

用简单模型实现高效精准的多模态3D目标跟踪,仅需260万参数。

RegTrack: Simplicity Beneath Complexity in Robust Multi-Modal 3D Multi-Object Tracking

  • 基于杨-米尔斯规范理论设计统一编码器,仅用点云输入完成跨帧关联
  • 通过自适应运动补偿提升鲁棒性,在KITTI和nuScenes上超越35种方法
  • 冻结视觉语言模型提供预训练表征,提升不同物体区分度与跨帧一致性

现有3D多目标跟踪方法常为追求鲁棒性牺牲效率与泛化能力,依赖复杂的多模态架构和类别相关运动先验。本文提出一种高效、鲁棒且泛化性强的多模态3D MOT方法RegTrack,其核心思想源于杨-米尔斯规范理论。RegTrack采用统一三线索编码器(UTEnc),包含局部-全局点云编码器(LG-PEnc)、基于专家混合的几何编码器(MoE-GEnc)以及来自预训练视觉-语言模型的图像编码器。LG-PEnc高效提取点云空间结构信息,其成对相似性作为唯一关联度量;MoE-GEnc与LG-PEnc协同建模跨帧物体间几何关系,自适应补偿运动变化,无需类别特定先验;图像编码器保持冻结,仅在训练中提供预训练表示空间,通过将点云表征对齐该空间,监督运动补偿过程,增强同一目标跨帧表征不变性与不同目标间可区分性。最终,仅使用点云输入即可实现鲁棒、高效、泛化的推理,参数量仅为2.67M。在KITTI和nuScenes上的大量实验表明,RegTrack优于35种对比方法。

原文摘要 · Abstract (English)

Existing 3D multi-object tracking (MOT) methods often sacrifice efficiency and generalizability for robustness, largely relying on complex association metrics derived from multi-modal architectures and class-specific motion priors. Challenging the rooted belief that greater complexity necessarily yields greater robustness, we propose a robust, efficient, and generalizable method for multi-modal 3D MOT, dubbed RegTrack. Inspired by Yang-Mills gauge theory, RegTrack is built upon a unified tri-cue encoder (UTEnc), comprising three tightly coupled components: a local-global point cloud encoder (LG-PEnc), a mixture-of-experts-based geometry encoder (MoE-GEnc), and an image encoder from a well-pretrained visual-language model. LG-PEnc efficiently encodes the spatial and structural information of point clouds to produce foundational representations for each object, whose pairwise similarities serve as the sole association metric. MoE-GEnc seamlessly interacts with LG-PEnc to model inter-object geometric relationships across frames, adaptively compensating for inter-frame object motion without relying on any class-specific priors. The image encoder is kept frozen and is used exclusively during training to provide a well-pretrained representation space. Point cloud representations are aligned to this space to supervise the motion compensation process, encouraging representation invariance across frames for the same object while enhancing discriminability among different objects. Through this formulation, RegTrack attains robust, efficient, and generalizable inference using only point cloud inputs, requiring just 2.6M parameters. Extensive experiments on KITTI and nuScenes show that RegTrack outperforms its thirty-five competitors.

3D跟踪点云处理运动补偿轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。