构建首个专家验证的孟加拉手语数据集，轻量模型实现高精度手机端识别
Toward Deployable Bangla Sign Language Recognition with Expert-Validated Data and a Lightweight Attention-Based Model
- 用分组残差块与注意力机制设计轻量网络，参数仅29万
- 在自建数据集上达96.37%准确率，比主流模型少68倍参数
- 可部署在手机上，0.48MB模型每帧处理仅3.98毫秒
孟加拉国聋哑人群主要使用孟加拉手语（BdSL）交流。在个人设备上实现自动BdSL识别可提升教育与服务可及性。现有系统依赖受控场景数据集且未经专家验证，采用过重的预训练主干网络，不适用于设备端部署。本文构建了RSBdSL38数据集，包含10,874张由专家验证的图像，覆盖全部38个BdSL手势，对应孟加拉字母表51个字母，拍摄于孟加拉三所特殊学校的真实手语者。提出一种轻量级注意力卷积网络，仅298,470参数，基于分组瓶颈残差块、通道与空间注意力、多尺度深度可分离手部特征块、双池化结构及Swish激活函数。从头训练，在相同评估协议下达到96.37%准确率（五次种子平均95.72%±0.54%），较九种ImageNet预训练高效架构最优结果仅低1.08个百分点，但参数减少8.5至68倍，乘法累加操作减少1.3至21.7倍。重新训练后，在六个公开BdSL基准上准确率达92.95%至98.33%，合并数据集上达97.04%，零样本测试在BdSL-38上为76.25%。移除任意网络模块导致性能下降7.61至89.30点，而训练策略影响最大仅3.17点。通过Grad-CAM删除-插入与权重随机化验证确认预测聚焦于手部动作。独立于签名者的划分中（保留6位签名者），准确率为85.18%。量化至0.48MB后，在普通智能手机上运行时延仅3.98毫秒，内存占用15.5MB。RSBdSL38数据集与从头训练模型共同将基准精度转化为实际可部署的无障碍访问，成本远低于预训练主干模型；数据集、代码与模型均已开源。
原文摘要 · Abstract (English)
Deaf and hard-of-hearing people in Bangladesh communicate mainly through Bangla Sign Language (BdSL). Automatic BdSL recognition on personal devices could widen access to education and services. Existing systems use controlled-setting datasets without expert verification and heavyweight pretrained backbones unsuited to on-device use. We introduce RSBdSL38, 10,874 expert-validated images spanning all 38 BdSL hand signs, representing the 51 letters of the Bangla alphabet, recorded from real signers at three special-needs schools across Bangladesh. We propose a lightweight attention based convolutional network of 298,470 parameters, built from grouped bottleneck residual blocks, channel and spatial attention, a multi-scale depthwise hand-feature block, dual pooling, and Swish activations. Trained from scratch, it attains 96.37% accuracy (95.72% +- 0.54% over five seeds), within 1.08 percentage points of the best of nine ImageNet-pretrained efficient architectures under an identical protocol, using 8.5 to 68x fewer parameters and 1.3 to 21.7x fewer MACs. Retrained, it reaches 92.95 to 98.33% on six public BdSL benchmarks, 97.04% on a merged corpus, and 76.25% zero-shot on BdSL-38. Removing any architectural stage costs 7.61 to 89.30 points, against at most 3.17 for the training recipe. Grad-CAM with deletion-insertion and weight-randomization checks confirms that predictions follow the signing hand. A signer-independent split holding out 6 of 36 signers yields 85.18%. Quantized to 0.48 MB, it runs at 3.98 ms per image within a 15.5 MB footprint on a commodity smartphone. Together, RSBdSL38 and our from-scratch model turn benchmark accuracy into deployable accessibility at a fraction of pretrained-backbone cost; dataset, code, and models are released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。