无需后处理，直接生成动作时序管，提升检测效率。
Action tube generation by person query matching for spatio-temporal action detection
- 用查询匹配机制在帧间追踪同一人，避免传统链接步骤。
- 在三个数据集上表现优异，尤其擅长处理大位移动作。
- 计算高效，资源占用低，适合长视频实时检测。
本文提出一种时空动作检测（STAD）方法，直接从原始视频生成动作时序管，无需依赖基于交并比的链接或片段拆分等后处理步骤。该方法对每帧应用基于查询的检测（DETR），并通过查询匹配将同一人物在不同帧中的查询关联起来。引入查询匹配模块（QMM），利用度量学习使同一人物的查询在跨帧间距离更近，而不同人物的查询则更远。通过QMM匹配获得的查询序列预测动作类别，支持长度可变的输入，适用于超过单个片段长度的视频。在JHMDB、UCF101-24和AVA数据集上的实验表明，该方法在人物大幅位移情况下表现良好，同时具备更高的计算效率和更低的资源需求。
原文摘要 · Abstract (English)
This paper proposes a method for spatio-temporal action detection (STAD) that directly generates action tubes from the original video without relying on post-processing steps such as IoU-based linking and clip splitting. Our approach applies query-based detection (DETR) to each frame and matches DETR queries to link the same person across frames. We introduce the Query Matching Module (QMM), which uses metric learning to bring queries for the same person closer together across frames compared to queries for different people. Action classes are predicted using the sequence of queries obtained from QMM matching, allowing for variable-length inputs from videos longer than a single clip. Experimental results on JHMDB, UCF101-24, and AVA datasets demonstrate that our method performs well for large position changes of people while offering superior computational efficiency and lower resource requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。