用轻量级神经网络实现高效手语识别，适合在低资源设备运行。
Sign Language Recognition using Bidirectional Reservoir Computing
- 结合MediaPipe与双向储备池计算，捕捉手势时间依赖性。
- 在WLASL数据集上达57.71%准确率，训练仅需9秒。
- 相比深度学习模型提速60倍，适合边缘设备部署。
手语识别（SLR）有助于聋人与听人之间的沟通。尽管深度学习广泛用于构建SLR系统，但其计算开销大，难以在资源受限设备上运行。为此，我们提出一种基于MediaPipe和基于回声状态网络（ESN）的双向储备池计算（BRC）架构的高效手语识别系统。MediaPipe提取手部关节坐标作为输入，BRC在前后两个方向处理特征，有效捕捉时间依赖性。BRC输出的状态拼接后形成分类用的鲁棒表示。我们在单词级美国手语（WLASL）视频数据集上评估该方法，取得57.71%的竞争力准确率，训练时间仅9秒，远低于深度学习的Bi-GRU方法所需的55分钟38秒。因此，该BRC-based SLR系统非常适合边缘设备部署。
原文摘要 · Abstract (English)
Sign language recognition (SLR) facilitates communication between deaf and hearing individuals. Deep learning is widely used to develop SLR-based systems; however, it is computationally intensive and requires substantial computational resources, making it unsuitable for resource-constrained devices. To address this, we propose an efficient sign language recognition system using MediaPipe and an echo state network (ESN)-based bidirectional reservoir computing (BRC) architecture. MediaPipe extracts hand joint coordinates, which serve as inputs to the ESN-based BRC architecture. The BRC processes these features in both forward and backward directions, efficiently capturing temporal dependencies. The resulting states of BRC are concatenated to form a robust representation for classification. We evaluated our method on the Word-Level American Sign Language (WLASL) video dataset, achieving a competitive accuracy of 57.71% and a significantly lower training time of only 9 seconds, in contrast to the 55 minutes and $38$ seconds required by the deep learning-based Bi-GRU approach. Consequently, the BRC-based SLR system is well-suited for edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。