首个俄语手语字母视频数据集，支持实时识别
Bukva: Russian Sign Language Alphabet
- 构建首个开源俄语手语字母动态视频数据集
- 达83.6%准确率，仅用CPU实现实时推理
- 面向残障人士与手语研究者，具实用价值
本文研究俄语手指拼写字母（即俄语手语，RSL dactyl）的识别。该方法用于拼写专有名词或术语，不依赖特定手语符号。现有俄语手指拼写数据集普遍存在样本不足、缺乏多样性或仅包含静态手势的问题。为此，我们推出Bukva，首个完整的开源视频数据集，包含3,757个视频，每种字母符号超过101个样本，涵盖动态手势。通过众包平台招募155名听障及重听专家参与录制，显著提升受试者多样性。采用时间移位模块（TSM）有效处理静态与动态手势，在仅使用CPU的情况下实现83.6%的top-1准确率，且支持实时推理。数据集、演示代码与预训练模型均已公开。
原文摘要 · Abstract (English)
This paper investigates the recognition of the Russian fingerspelling alphabet, also known as the Russian Sign Language (RSL) dactyl. Dactyl is a component of sign languages where distinct hand movements represent individual letters of a written language. This method is used to spell words without specific signs, such as proper nouns or technical terms. The alphabet learning simulator is an essential isolated dactyl recognition application. There is a notable issue of data shortage in isolated dactyl recognition: existing Russian dactyl datasets lack subject heterogeneity, contain insufficient samples, or cover only static signs. We provide Bukva, the first full-fledged open-source video dataset for RSL dactyl recognition. It contains 3,757 videos with more than 101 samples for each RSL alphabet sign, including dynamic ones. We utilized crowdsourcing platforms to increase the subject's heterogeneity, resulting in the participation of 155 deaf and hard-of-hearing experts in the dataset creation. We use a TSM (Temporal Shift Module) block to handle static and dynamic signs effectively, achieving 83.6% top-1 accuracy with a real-time inference with CPU only. The dataset, demo code, and pre-trained models are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。