解决微动作识别标注难问题,用异步伪标签提升模型精度
Towards Micro-Action Recognition with Limited Annotations: An Asynchronous Pseudo Labeling and Training Approach
- 分离伪标签生成与训练过程,避免错误累积
- 通过聚类和自适应阈值过滤噪声标签,提升伪标签质量
- 适合标注数据少的微动作识别场景,尤其适合资源受限任务
微动作识别(MAR)旨在分类视频中细微的人体动作,但因动作细微难以标注。为此,本文提出半监督微动作识别(SSMAR)设定,仅部分样本有标签。我们评估了传统半监督学习方法在该任务上的表现,发现其易受不准确伪标签影响,导致误差积累、性能下降。根源在于直接将分类器输出作为伪标签进行训练。为此,我们提出异步伪标签与训练框架(APLT),在离线阶段使用半监督聚类生成更准确的伪标签,并引入自适应阈值策略动态过滤不同类别中的噪声标签。随后构建基于过滤后伪标签的记忆原型分类器,固定不变以指导后续训练。通过异步交替伪标签生成与模型训练,模型能学习更高质量伪标签,同时避免过拟合。在三个MAR数据集上的实验表明,APLT显著优于现有SOTA方法。例如,在仅使用50%标注数据时,相比FixMatch在MA-12数据集上提升准确率14.5%。
原文摘要 · Abstract (English)
Micro-Action Recognition (MAR) aims to classify subtle human actions in video. However, annotating MAR datasets is particularly challenging due to the subtlety of actions. To this end, we introduce the setting of Semi-Supervised MAR (SSMAR), where only a part of samples are labeled. We first evaluate traditional Semi-Supervised Learning (SSL) methods to SSMAR and find that these methods tend to overfit on inaccurate pseudo-labels, leading to error accumulation and degraded performance. This issue primarily arises from the common practice of directly using the predictions of classifier as pseudo-labels to train the model. To solve this issue, we propose a novel framework, called Asynchronous Pseudo Labeling and Training (APLT), which explicitly separates the pseudo-labeling process from model training. Specifically, we introduce a semi-supervised clustering method during the offline pseudo-labeling phase to generate more accurate pseudo-labels. Moreover, a self-adaptive thresholding strategy is proposed to dynamically filter noisy labels of different classes. We then build a memory-based prototype classifier based on the filtered pseudo-labels, which is fixed and used to guide the subsequent model training phase. By alternating the two pseudo-labeling and model training phases in an asynchronous manner, the model can not only be learned with more accurate pseudo-labels but also avoid the overfitting issue. Experiments on three MAR datasets show that our APLT largely outperforms state-of-the-art SSL methods. For instance, APLT improves accuracy by 14.5\% over FixMatch on the MA-12 dataset when using only 50\% labeled data. Code will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。