用Transformer统一多摄像头点跟踪，兼顾外观与几何信息。
Look Around and Pay Attention: Multi-camera Point Tracking Reimagined with Transformers
- 通过跨视角注意力融合外观与几何先验，建立软对应关系。
- 在TAPVid-3D-MC上达37.5% APD，PointOdyssey-MC上达90.3% APD。
- 适合处理复杂运动与遮挡场景，对轨迹一致性有显著提升。
本文提出LAPA（Look Around and Pay Attention），一种基于Transformer的端到端多摄像头点跟踪架构，将外观匹配与几何约束融合。传统方法分步处理检测、关联与跟踪，易导致误差传播和时间不一致。LAPA利用注意力机制联合跨视图与时间推理，通过增强几何先验的跨视角注意力建立软对应关系。不依赖传统三角化，而是通过注意力加权聚合构建3D点表示，自然处理不确定性与部分观测。通过Transformer解码器建模长时依赖，保持长时间遮挡下的身份一致性。在挑战性数据集上进行大量实验，包括新创建的TAPVid-3D全景与PointOdyssey的多摄像头版本，结果表明该统一方法显著优于现有方法，在TAPVid-3D-MC上达到37.5% APD，PointOdyssey-MC上达90.3% APD，尤其在复杂运动与遮挡场景表现优异。代码已开源。
原文摘要 · Abstract (English)
This paper presents LAPA (Look Around and Pay Attention), a novel end-to-end transformer-based architecture for multi-camera point tracking that integrates appearance-based matching with geometric constraints. Traditional pipelines decouple detection, association, and tracking, leading to error propagation and temporal inconsistency in challenging scenarios. LAPA addresses these limitations by leveraging attention mechanisms to jointly reason across views and time, establishing soft correspondences through a cross-view attention mechanism enhanced with geometric priors. Instead of relying on classical triangulation, we construct 3D point representations via attention-weighted aggregation, inherently accommodating uncertainty and partial observations. Temporal consistency is further maintained through a transformer decoder that models long-range dependencies, preserving identities through extended occlusions. Extensive experiments on challenging datasets, including our newly created multi-camera (MC) versions of TAPVid-3D panoptic and PointOdyssey, demonstrate that our unified approach significantly outperforms existing methods, achieving 37.5% APD on TAPVid-3D-MC and 90.3% APD on PointOdyssey-MC, particularly excelling in scenarios with complex motions and occlusions. Code is available at https://github.com/ostadabbas/Look-Around-and-Pay-Attention-LAPA-
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。