无需英语中介，直接将多语言转为手语，支持实时生成。
CSF: Contrastive Semantic Features for Direct Multilingual Sign Language Generation
- 用9个通用语义槽分解句子，构建无语言依赖的语义表示。
- 在4种语言上槽提取准确率达99.03%，条件分类达99.4%。
- 模型轻量（0.74MB），CPU推理仅3.02ms，适合浏览器实时应用。
手语翻译系统通常需要英语作为中间语言，对全球聋人群体中的非英语使用者构成障碍。本文提出一种无语言依赖的语义表征框架——规范语义形式（CSF），实现任意源语言到手语的直接转换。CSF将话语分解为九个通用语义槽：事件、意图、时间、条件、施事者、受事者、位置、目的和修饰语。关键贡献是包含35种条件类型的完整条件分类体系，覆盖八个语义类别，可细致表达日常交流中的条件表达。我们训练了一个轻量级Transformer提取器（0.74MB），在四种语言（英语、越南语、日语、法语）上平均槽提取准确率达99.03%，尤其在复杂35类条件分类中达到99.4%准确率。模型在CPU上推理延迟仅为3.02ms，支持浏览器内实时手语生成。代码、训练模型及多语言数据集已开源，助力无障碍手语技术研究。
原文摘要 · Abstract (English)
Sign language translation systems typically require English as an intermediary language, creating barriers for non-English speakers in the global deaf community. We present Canonical Semantic Form (CSF), a language-agnostic semantic representation framework that enables direct translation from any source language to sign language without English mediation. CSF decomposes utterances into nine universal semantic slots: event, intent, time, condition, agent, object, location, purpose, and modifier. A key contribution is our comprehensive condition taxonomy comprising 35 condition types across eight semantic categories, enabling nuanced representation of conditional expressions common in everyday communication. We train a lightweight transformer-based extractor (0.74 MB) that achieves 99.03% average slot extraction accuracy across four typologically diverse languages: English, Vietnamese, Japanese, and French. The model demonstrates particularly strong performance on condition classification (99.4% accuracy) despite the 35-class complexity. With inference latency of 3.02ms on CPU, our approach enables real-time sign language generation in browser-based applications. We release our code, trained models, and multilingual dataset to support further research in accessible sign language technology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。