用少量数据训练轻量模型，实现毫秒级手语实时识别
Developing Lightweight DNN Models With Limited Data For Real-Time Sign Language Recognition
- 将手形、朝向、动作等手语特征转为向量输入，结合MediaPipe提取关键点
- 模型仅10MB以下，在边缘设备上识别343个手势准确率达92%，延迟低于10ms
- 配套标注平台和网页应用已上线，适合移动端手语识别场景
我们提出一种基于轻量级DNN的手语实时识别框架，针对数据稀缺、计算成本高及训练与推理帧率不一致等挑战。通过将手形、掌面朝向、运动轨迹和位置等手语特征求向量化，并利用MediaPipe提取关键点，获得高度可分的输入表示。所设计的DNN架构优化后可小于10MB部署，能在边缘设备上实现343个孤立手势的准确分类，推理延迟低于10ms。配套开发的数据标注平台' slait data '支持结构化标注与向量提取。模型在孤立手语识别中达到92%准确率，并已集成至' slait ai '网页应用，展现出稳定的推理性能。
原文摘要 · Abstract (English)
We present a novel framework for real-time sign language recognition using lightweight DNNs trained on limited data. Our system addresses key challenges in sign language recognition, including data scarcity, high computational costs, and discrepancies in frame rates between training and inference environments. By encoding sign language specific parameters, such as handshape, palm orientation, movement, and location into vectorized inputs, and leveraging MediaPipe for landmark extraction, we achieve highly separable input data representations. Our DNN architecture, optimized for sub 10MB deployment, enables accurate classification of 343 signs with less than 10ms latency on edge devices. The data annotation platform 'slait data' facilitates structured labeling and vector extraction. Our model achieved 92% accuracy in isolated sign recognition and has been integrated into the 'slait ai' web application, where it demonstrates stable inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。