用轻量级计算实现手语实时识别，适合在边缘设备部署。
Sign Language Recognition using Parallel Bidirectional Reservoir Computing
- 并行双向时滞网络结合手势追踪数据，捕捉动作时序特征。
- 在WLASL数据集上达到60.85%准确率，训练仅需18.67秒。
- 无需复杂模型，适合低资源设备上的实时手语识别应用。
手语识别（SLR）促进聋人与听力人士沟通。现有深度学习方法虽有效但计算开销大，难以在边缘设备部署。为此，我们提出一种轻量级SLR系统，结合并行双向时滞网络（PBRC）与MediaPipe。MediaPipe实现实时手部追踪，精准提取手关节坐标作为输入特征。PBRC由两个基于回声状态网络的双向时滞模块并行构成，有效捕捉时间依赖性，生成丰富特征表示用于分类。在单词级美式手语（WLASL）视频数据集上训练，取得60.85%、85.86%和91.74%的Top-1、Top-5、Top-10准确率。由于时滞计算固有特性，训练时间仅18.67秒，远低于深度学习方法如Bi-GRU所需的55分钟以上。该方案为边缘设备上的实时手语识别提供高效低成本解决方案。
原文摘要 · Abstract (English)
Sign language recognition (SLR) facilitates communication between deaf and hearing communities. Deep learning based SLR models are commonly used but require extensive computational resources, making them unsuitable for deployment on edge devices. To address these limitations, we propose a lightweight SLR system that combines parallel bidirectional reservoir computing (PBRC) with MediaPipe. MediaPipe enables real-time hand tracking and precise extraction of hand joint coordinates, which serve as input features for the PBRC architecture. The proposed PBRC architecture consists of two echo state network (ESN) based bidirectional reservoir computing (BRC) modules arranged in parallel to capture temporal dependencies, thereby creating a rich feature representation for classification. We trained our PBRC-based SLR system on the Word-Level American Sign Language (WLASL) video dataset, achieving top-1, top-5, and top-10 accuracies of 60.85%, 85.86%, and 91.74%, respectively. Training time was significantly reduced to 18.67 seconds due to the intrinsic properties of reservoir computing, compared to over 55 minutes for deep learning based methods such as Bi-GRU. This approach offers a lightweight, cost-effective solution for real-time SLR on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。