提出通用手语转写系统,实现多语言手语实时处理
Real-Time Multilingual Sign Language Processing
- 用SignWiring统一转写手语,连接视觉动作与文本
- 验证转写法提升跨语言手语翻译速度与准确性
- 为聋哑人群提供实时多语言手语技术新可能
手语处理(SLP)是自然语言处理与计算机视觉的交叉领域,旨在实现手语的计算理解、翻译与生成。传统方法受限于依赖词素的特定语言系统,难以捕捉手语的多维特征,阻碍了有效技术的发展。本文提出一种新范式,采用SignWiring这一通用手语转写符号系统,作为手语视觉手势与文本语言表示之间的中介。我们贡献了基础库与资源,推动手语翻译与生成任务的研究。这些任务包括从视频到口语文本的双向转换。实证评估表明,该转写方法能加速、精准地支持多语言手语研究,实现更自然准确的翻译。其通用性为实时多语言手语应用铺平道路,助力构建更包容、可及的语言科技,使更多聋哑人群受益。
原文摘要 · Abstract (English)
Sign Language Processing (SLP) is an interdisciplinary field comprised of Natural Language Processing (NLP) and Computer Vision. It is focused on the computational understanding, translation, and production of signed languages. Traditional approaches have often been constrained by the use of gloss-based systems that are both language-specific and inadequate for capturing the multidimensional nature of sign language. These limitations have hindered the development of technology capable of processing signed languages effectively. This thesis aims to revolutionize the field of SLP by proposing a simple paradigm that can bridge this existing technological gap. We propose the use of SignWiring, a universal sign language transcription notation system, to serve as an intermediary link between the visual-gestural modality of signed languages and text-based linguistic representations. We contribute foundational libraries and resources to the SLP community, thereby setting the stage for a more in-depth exploration of the tasks of sign language translation and production. These tasks encompass the translation of sign language from video to spoken language text and vice versa. Through empirical evaluations, we establish the efficacy of our transcription method as a pivot for enabling faster, more targeted research, that can lead to more natural and accurate translations across a range of languages. The universal nature of our transcription-based paradigm also paves the way for real-time, multilingual applications in SLP, thereby offering a more inclusive and accessible approach to language technology. This is a significant step toward universal accessibility, enabling a wider reach of AI-driven language technologies to include the deaf and hard-of-hearing community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。