通过关注身体部位提升德语手语同音词辨析准确率
Sign Language Sense Disambiguation
- 用Transformer模型聚焦手部或口部特征进行辨析
- 小数据下关注口部提升性能，大数据下手部更优
- 适合手语识别系统开发者和无障碍技术研究者
本项目探索提升德语手语翻译中同音词歧义消解的方法。手语具有模糊性且研究较少，为此我们采用基于Transformer的模型，对不同身体部位表征进行训练，以聚焦特定部位。通过实验不同组合，发现小样本场景下聚焦口部可提升性能，大样本场景下聚焦手部效果更佳。该成果有助于改善数字助手等系统的准确性，增强聋人用户的交互体验。项目代码已开源。
原文摘要 · Abstract (English)
This project explores methods to enhance sign language translation of German sign language, specifically focusing on disambiguation of homonyms. Sign language is ambiguous and understudied which is the basis for our experiments. We approach the improvement by training transformer-based models on various bodypart representations to shift the focus on said bodypart. To determine the impact of, e.g., the hand or mouth representations, we experiment with different combinations. The results show that focusing on the mouth increases the performance in small dataset settings while shifting the focus on the hands retrieves better results in larger dataset settings. Our results contribute to better accessibility for non-hearing persons by improving the systems powering digital assistants, enabling a more accurate interaction. The code for this project can be found on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。