对比两种手语的象形性对迁移学习效果的影响
The Influence of Iconicity in Transfer Learning for Sign Language Recognition
- 用象形性相似的手语对做迁移学习对比实验
- 阿拉伯语手语提升7.02%，弗拉芒语手语提升1.07%
- 适合手语识别与跨语言迁移研究者阅读
大多数手语识别研究依赖于从ImageNet等视觉数据集进行迁移学习（TL）。部分研究尝试使用其他语言数据集，常聚焦于具有跨语言相似性的手语。本文通过比较两种不同手语对（中文→阿拉伯语、希腊语→弗拉芒语）中象形性手语的迁移学习表现，检验这些相似性在有效知识迁移中的必要性。采用Google Mediapipe作为输入特征提取器，利用多层感知机处理空间信息，门控循环单元处理时间信息。实验结果显示，从中文向阿拉伯语迁移时性能提升7.02%，从希腊语向弗拉芒语迁移时提升1.07%。
原文摘要 · Abstract (English)
Most sign language recognition research relies on Transfer Learning (TL) from vision-based datasets such as ImageNet. Some extend this to alternatively available language datasets, often focusing on signs with cross-linguistic similarities. This body of work examines the necessity of these likenesses on effective knowledge transfer by comparing TL performance between iconic signs of two different sign language pairs: Chinese to Arabic and Greek to Flemish. Google Mediapipe was utilised as an input feature extractor, enabling spatial information of these signs to be processed with a Multilayer Perceptron architecture and the temporal information with a Gated Recurrent Unit. Experimental results showed a 7.02% improvement for Arabic and 1.07% for Flemish when conducting iconic TL from Chinese and Greek respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。