在FPGA上实现低功耗实时多目标追踪，结合量化YOLOv8与SORT算法
Real-Time Multi-Object Tracking using YOLOv8 and SORT on a SoC FPGA
- 用量化YOLOv8做检测，SORT做追踪，全系统部署在Zynq SoC FPGA上
- 在MOT15数据集上达38.9% MOTA，COCO上0.21 mAP，满足实时性要求
- 适合嵌入式机器人感知系统，尤其关注低功耗与实时性能的开发者
多目标追踪（MOT）是计算机视觉中的关键问题，也是高级自主移动机器人视觉感知系统的核心组件。为实现低功耗、实时运行，本文提出一种基于量化YOLOv8检测器与SORT追踪器的FPGA嵌入式MOT系统。采用改进版FINN框架，利用外部内存存储模型参数，并支持YOLOv8所需运算。在AMD/Xilinx Zynq UltraScale+ MPSoC平台上，检测模块部署于可重构逻辑，追踪算法运行于处理器系统。使用COCO和MOT15数据集评估性能，获得0.21 mAP和38.9 MOTA。该方案显著降低计算与内存开销，适用于资源受限的实时视觉系统。
原文摘要 · Abstract (English)
Multi-object tracking (MOT) is one of the most important problems in computer vision and a key component of any vision-based perception system used in advanced autonomous mobile robotics. Therefore, its implementation on low-power and real-time embedded platforms is highly desirable. Modern MOT algorithms should be able to track objects of a given class (e.g. people or vehicles). In addition, the number of objects to be tracked is not known in advance, and they may appear and disappear at any time, as well as be obscured. For these reasons, the most popular and successful approaches have recently been based on the tracking paradigm. Therefore, the presence of a high quality object detector is essential, which in practice accounts for the vast majority of the computational and memory complexity of the whole MOT system. In this paper, we propose an FPGA (Field-Programmable Gate Array) implementation of an embedded MOT system based on a quantized YOLOv8 detector and the SORT (Simple Online Realtime Tracker) tracker. We use a modified version of the FINN framework to utilize external memory for model parameters and to support operations necessary required by YOLOv8. We discuss the evaluation of detection and tracking performance using the COCO and MOT15 datasets, where we achieve 0.21 mAP and 38.9 MOTA respectively. As the computational platform, we use an MPSoC system (Zynq UltraScale+ device from AMD/Xilinx) where the detector is deployed in reprogrammable logic and the tracking algorithm is implemented in the processor system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。