用3D CNN实现实时手部手势识别，应对动作差异大和低延迟挑战
Online Hand Gesture Recognition Using 3D Convolutional Neural Networks
- 基于3D卷积网络结合滑动窗口，提升实时手势定位与分类鲁棒性
- 在Jester数据集上检测器准确率超98%，分类器超90%
- 系统可在3秒内响应，自建数据集上达到37.5%的Levenshtein准确率
在人机交互中，动态手部手势的实时检测与分类具有挑战性：一方面系统需在视频流中实时运行，操作后无明显延迟；另一方面个体间手势表现差异大，增加识别难度。本文提出一种在线手部手势识别系统，可实时定位视频流中的手势并识别其内容。为增强系统鲁棒性，采用滑动窗口策略融合多窗口结果。所有模型均在Jester数据集上训练，检测器准确率超过98%，分类器准确率超过90%。整体系统在自建数据集上最佳组别响应时间在三秒内，Levenshtein准确率达37.5%。项目代码已公开。
原文摘要 · Abstract (English)
In human computer interaction, real-time detection and classification of dynamic hand gestures is challenging as: 1) the system must run in a real-time video stream and there is no noticeable lag in response after performing a gesture; 2) there is a large difference in how people perform gestures, making recognition more difficult. In this paper, an online hand gesture recognition system is proposed, which is able to localize gestures in real-time video stream and recognize what these gestures are. To improve the robustness of the system, the sliding window approach is used to refine results from multiple windows. All of the models in my project are trained on Jester database, achieving 98+% accuracy for detector and 90+% accuracy for classifier. For the overall performance of the system, the best group can respond within three seconds and reach 37.5% Levenshtein accuracy on the homemade dataset. The project codes used in this work are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。