一次搞定多人重复动作检测、追踪与计数，实时高效。
MultiCounter: Multiple Action Agnostic Repetition Counting in Untrimmed Videos
- 端到端框架同步处理多人体重复动作的检测、追踪和计数。
- 在MultiRep数据集上，周期性mAP提升41.0%，平均绝对误差降58.6%。
- 不依赖人体数量，适合体育、健身等真实场景应用。
多实例重复动作计数（MRAC）旨在估计未剪辑视频中多个个体执行的重复动作次数，常见于体育和健身等以人为中心的领域。本文提出一种全端到端的深度学习框架MultiCounter，可同时实现多人体重复动作的检测、追踪与计数。具体地，MultiCounter引入两个新模块：1）混合时空交互，高效捕捉连续帧间的上下文关联；2）任务专用头，精准感知周期边界并实现对无特定动作的人体实例的泛化能力。模型在自动生成的合成数据集MultiRep上训练，该数据集基于真实视频标注生成。在MultiRep上的实验验证了MRAC任务的根本挑战，并展示了所提模型的优越性。相比ByteTrack+RepNet（先进追踪器+单一计数器组合），MultiCounter在周期性mAP上提升41.0%，平均绝对误差（AvgMAE）降低58.6%，平均超出边界率（AvgOBO）提高1.48倍。该方法在实时性上表现优异，可在普通GPU服务器上运行，且对视频中人体数量不敏感。
原文摘要 · Abstract (English)
Multi-instance Repetitive Action Counting (MRAC) aims to estimate the number of repetitive actions performed by multiple instances in untrimmed videos, commonly found in human-centric domains like sports and exercise. In this paper, we propose MultiCounter, a fully end-to-end deep learning framework that enables simultaneous detection, tracking, and counting of repetitive actions of multiple human instances. Specifically, MultiCounter incorporates two novel modules: 1) mixed spatiotemporal interaction for efficient context correlation across consecutive frames, and 2) task-specific heads for accurate perception of periodic boundaries and generalization for action-agnostic human instances. We train MultiCounter on a synthetic dataset called MultiRep generated from annotated real-world videos. Experiments on the MultiRep dataset validate the fundamental challenge of MRAC tasks and showcase the superiority of our proposed model. Compared to ByteTrack+RepNet, a solution that combines an advanced tracker with a single repetition counter, MultiCounter substantially improves Period-mAP by 41.0%, reduces AvgMAE by 58.6%, and increases AvgOBO 1.48 times. This sets a new benchmark in the field of MRAC. Moreover, MultiCounter runs in real-time on a commodity GPU server and is insensitive to the number of human instances in a video.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。