arXiv:2411.06780cs.CV2024-11NeurIPS被引 2

提出SynCL策略,解决多摄像头3D跟踪中检测与跟踪的协同优化难题。

SynCL: A Synergistic Training Strategy with Instance-Aware Contrastive Learning for End-to-End Multi-Camera 3D Tracking

论文配图:SynCL: A Synergistic Training Strategy with Instance-Aware Contrastive Learning for End-to-End Multi-Camera 3D Tracking
图 1 · 摘自论文原文
  • 设计任务特异性匹配模块,突破自注意力机制对候选框的过度去重限制。
  • 引入动态查询过滤与实例感知对比学习,提升跟踪与检测协同效果。
  • 无需额外推理开销,在nuScenes上达58.9% AMOTA,适合多摄像头3D跟踪研究者。

现有基于查询的端到端3D视觉跟踪器通过注意力机制整合检测与跟踪,但二者共享参数时面临优化困境。研究发现,这源于自注意力机制的两个固有缺陷:对象查询的过度去重和轨迹查询的自我中心注意力。相反,移除自注意力机制不仅对跟踪回归预测影响极小,还倾向于生成更多潜在候选框。基于此,本文提出SynCL——一种即插即用的协同训练策略,用于促进检测与跟踪的多任务学习。具体地,设计任务特异性混合匹配模块,使轨迹查询与多个对象查询匹配,挖掘被自注意力忽略的优质候选;为灵活选择最优候选,引入由模型训练状态控制的动态查询过滤模块;并提出实例感知对比学习,打破轨迹查询的自我中心注意力瓶颈,有效弥合检测与跟踪差距。在不增加推理开销的前提下,SynCL在多个基准上持续提升性能,在nuScenes数据集上达到58.9% AMOTA的当前最优水平。代码与原始结果将公开。

原文摘要 · Abstract (English)

While existing query-based 3D end-to-end visual trackers integrate detection and tracking via the tracking-by-attention paradigm, these two chicken-and-egg tasks encounter optimization difficulties when sharing the same parameters. Our findings reveal that these difficulties arise due to two inherent constraints on the self-attention mechanism, i.e., over-deduplication for object queries and self-centric attention for track queries. In contrast, removing the self-attention mechanism not only minimally impacts regression predictions of the tracker, but also tends to generate more latent candidate boxes. Based on these analyses, we present SynCL, a novel plug-and-play synergistic training strategy designed to co-facilitate multi-task learning for detection and tracking. Specifically, we propose a Task-specific Hybrid Matching module for a weight-shared cross-attention-based decoder that matches the targets of track queries with multiple object queries to exploit promising candidates overlooked by the self-attention mechanism. To flexibly select optimal candidates for the one-to-many matching, we also design a Dynamic Query Filtering module controlled by model training status. Moreover, we introduce Instance-aware Contrastive Learning to break through the barrier of self-centric attention for track queries, effectively bridging the gap between detection and tracking. Without additional inference costs, SynCL consistently delivers improvements in various benchmarks and achieves state-of-the-art performance with $58.9\%$ AMOTA on the nuScenes dataset. Code and raw results will be publicly available.

3D跟踪协同训练对比学习多摄像头

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。