无需标注数据,用图网络自动定位运动动作,速度快精度高。
UTAL-GNN: Unsupervised Temporal Action Localization using Graph Neural Networks
- 用图网络学习人体姿态序列的运动规律,无须人工标注。
- 在跳水数据集上达82.66%精度,定位延迟仅29毫秒。
- 可直接应用于真实场景视频,适合嵌入式实时系统。
在未剪辑的体育视频中进行细粒度动作定位面临挑战,因动作变化迅速且细微。现有监督与弱监督方法依赖大量标注数据和大模型,计算开销大,难以适应真实场景。本文提出一种轻量级、基于骨架的无监督动作定位流程,利用时空图神经网络表示。通过在块划分的姿态序列去噪任务上预训练注意力时空图卷积网络(ASTGCN),使其无需标注即可学习内在运动动态。推理时,定义一种新的动作动力学度量(ADM),直接从低维嵌入计算,通过检测其曲率拐点识别运动边界。该方法在DSV跳水数据集上达到82.66%的平均精度(mAP)和29.09毫秒的平均定位延迟,性能媲美最先进监督方法,同时保持高效计算。此外,无需重训练即可泛化到未见过的真实跳水视频,证明其在嵌入式或动态环境中的实际应用潜力。
原文摘要 · Abstract (English)
Fine-grained action localization in untrimmed sports videos presents a significant challenge due to rapid and subtle motion transitions over short durations. Existing supervised and weakly supervised solutions often rely on extensive annotated datasets and high-capacity models, making them computationally intensive and less adaptable to real-world scenarios. In this work, we introduce a lightweight and unsupervised skeleton-based action localization pipeline that leverages spatio-temporal graph neural representations. Our approach pre-trains an Attention-based Spatio-Temporal Graph Convolutional Network (ASTGCN) on a pose-sequence denoising task with blockwise partitions, enabling it to learn intrinsic motion dynamics without any manual labeling. At inference, we define a novel Action Dynamics Metric (ADM), computed directly from low-dimensional ASTGCN embeddings, which detects motion boundaries by identifying inflection points in its curvature profile. Our method achieves a mean Average Precision (mAP) of 82.66% and average localization latency of 29.09 ms on the DSV Diving dataset, matching state-of-the-art supervised performance while maintaining computational efficiency. Furthermore, it generalizes robustly to unseen, in-the-wild diving footage without retraining, demonstrating its practical applicability for lightweight, real-time action analysis systems in embedded or dynamic environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。