用关键点+卷积网络识别巴西手语手势，实时准确率达95%
Dynamic LIBRAS Gesture Recognition via CNN over Spatiotemporal Matrix Representation
- 通过媒体管道提取21个手部关键点，构建时空矩阵输入CNN
- 在低光下达95%准确率，正常光下92%，支持连续识别
- 适合智能家居中手势控制，尤其适用于手语识别场景
本文提出一种基于双模型组合的动态手势识别方法：使用MediaPipe手部关键点检测器提取21个手部骨骼关键点，并将其转化为90×21的时空矩阵，输入卷积神经网络（CNN）进行手势分类。该方法应用于家庭自动化系统中巴西手语（LIBRAS）手势识别，涵盖11类静态与动态手势。为实现实时推理，采用带帧复制的滑动窗口机制，无需循环网络即可连续识别。测试结果显示，在低光条件下准确率达95%，正常光照下为92%。结果表明该方法有效，但需更广泛的用户多样性实验以充分评估泛化性能。
原文摘要 · Abstract (English)
This paper proposes a method for dynamic hand gesture recognition based on the composition of two models: the MediaPipe Hand Landmarker, responsible for extracting 21 skeletal keypoints of the hand, and a convolutional neural network (CNN) trained to classify gestures from a spatiotemporal matrix representation of dimensions 90 by 21 of those keypoints. The method is applied to the recognition of LIBRAS (Brazilian Sign Language) gestures for device control in a home automation system, covering 11 classes of static and dynamic gestures. For real-time inference, a sliding window with temporal frame triplication is used, enabling continuous recognition without recurrent networks. Tests achieved 95\% accuracy under low-light conditions and 92\% under normal lighting. The results indicate that the approach is effective, although systematic experiments with greater user diversity are needed for a more thorough evaluation of generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。