用普通摄像头实现多目标3D轨迹追踪，精度高且无需人工标定。
A Modular Pipeline for 3D Object Tracking Using RGB Cameras
- 模块化设计，融合检测与滤波，支持多摄像头同步输入。
- 在987万帧中准确追踪小物体，3分钟实验平均处理11万像素点。
- 自动处理物体出现消失，适合大规模无标定场景的跟踪任务。
物体追踪是计算机视觉的关键挑战，广泛应用需不同架构。现有系统常受限于2D平面，且仅跟踪单个物体。本文提出一种新模块化流水线，可计算多个物体的3D轨迹。适用于多台时间同步、位置固定的摄像头（如6个普通网络摄像头）记录移动物体的场景。在桌布布置数据集上测试，参与者使用多种传感器记录，需追踪餐具类操作物体。挑战包括：在9,874,699帧中检测小物体、确定相机位姿、区分邻近重叠物体、应对临时遮挡，以及基于每3分钟试验平均11,124,56个像素坐标的子集计算3D轨迹。实现稳健流水线，生成带位置协方差（x,y,z）的置信度指标。动态处理物体出现与消失，实时启动新的扩展卡尔曼滤波器。可扩展至数百次桌布布置试验，几乎无需人工标注，即使每轮试验相机位姿未知。代码已开源。
原文摘要 · Abstract (English)
Object tracking is a key challenge of computer vision with various applications that all require different architectures. Most tracking systems have limitations such as constraining all movement to a 2D plane and they often track only one object. In this paper, we present a new modular pipeline that calculates 3D trajectories of multiple objects. It is adaptable to various settings where multiple time-synced and stationary cameras record moving objects, using off the shelf webcams. Our pipeline was tested on the Table Setting Dataset, where participants are recorded with various sensors as they set a table with tableware objects. We need to track these manipulated objects, using 6 rgb webcams. Challenges include: Detecting small objects in 9.874.699 camera frames, determining camera poses, discriminating between nearby and overlapping objects, temporary occlusions, and finally calculating a 3D trajectory using the right subset of an average of 11.12.456 pixel coordinates per 3-minute trial. We implement a robust pipeline that results in accurate trajectories with covariance of x,y,z-position as a confidence metric. It deals dynamically with appearing and disappearing objects, instantiating new Extended Kalman Filters. It scales to hundreds of table-setting trials with very little human annotation input, even with the camera poses of each trial unknown. The code is available at https://github.com/LarsBredereke/object_tracking
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。