无需真实数据,通过原型引导伪特征重放实现手势识别持续学习。
Data-Free Class-Incremental Gesture Recognition with Prototype-Guided Pseudo Feature Replay
- 用类别原型生成动态伪特征,避免遗忘旧手势
- 在两个数据集上平均准确率提升12.3%,优于现有方法
- 适合无数据访问限制的长期手势识别系统
手势识别是计算机视觉的重要研究方向。多数工作聚焦于闭集场景,难以应对新出现的手势。本文提出一种无数据类增量手势识别框架 PGPFR,包含四部分:基于批次原型的伪特征生成(PFGBP)、旧类变分原型重放(VPR)、新类截断交叉熵(TCE)和持续分类器重训练(CCRT)。PFGBP利用旧类原型与新类批次原型在线生成多样伪特征;VPR通过原型与协方差矩阵保持分类器权重一致性,增强鲁棒性;TCE缓解伪特征带来的领域差异影响;CCRT防止对新类过拟合,保障旧类特征稳定性。在SHREC 2017 3D和EgoGesture 3D两个常用数据集上的实验表明,本方法分别以11.8%和12.8%的均全局准确率提升超越现有最优方法。代码已开源。
原文摘要 · Abstract (English)
Gesture recognition is an important research area in the field of computer vision. Most gesture recognition efforts focus on close-set scenarios, thereby limiting the capacity to effectively handle unseen or novel gestures. We aim to address class-incremental gesture recognition, which entails the ability to accommodate new and previously unseen gestures over time. Specifically, we introduce a Prototype-Guided Pseudo Feature Replay (PGPFR) framework for data-free class-incremental gesture recognition. This framework comprises four components: Pseudo Feature Generation with Batch Prototypes (PFGBP), Variational Prototype Replay (VPR) for old classes, Truncated Cross-Entropy (TCE) for new classes, and Continual Classifier Re-Training (CCRT). To tackle the issue of catastrophic forgetting, the PFGBP dynamically generates a diversity of pseudo features in an online manner, leveraging class prototypes of old classes along with batch class prototypes of new classes. Furthermore, the VPR enforces consistency between the classifier's weights and the prototypes of old classes, leveraging class prototypes and covariance matrices to enhance robustness and generalization capabilities. The TCE mitigates the impact of domain differences of the classifier caused by pseudo features. Finally, the CCRT training strategy is designed to prevent overfitting to new classes and ensure the stability of features extracted from old classes. Extensive experiments conducted on two widely used gesture recognition datasets, namely SHREC 2017 3D and EgoGesture 3D, demonstrate that our approach outperforms existing state-of-the-art methods by 11.8\% and 12.8\% in terms of mean global accuracy, respectively. The code is available on https://github.com/sunao-101/PGPFR-3/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。