对比Transformer与LSTM模型，提升手语转文本准确率
American Sign Language to Text Translation using Transformer and Seq2Seq with LSTM
- 用Transformer替代Seq2Seq提升手语翻译性能
- Transformer相比Seq2Seq BLEU得分提升28.14点
- 在Transformer中加入ResidualLSTM反而使性能下降23.37%
手语翻译是聋人与听人沟通的重要课题，通过手势、身体和口部动作表达语言。美国手语（ASL）包含字母手语等类型。神经机器翻译技术正向手语翻译发展。变压器（Transformer）已成为自然语言处理的领先模型。本研究比较了Transformer与序列到序列（Seq2Seq）模型在手语转文本任务中的表现，并在Transformer中引入残差长短期记忆网络（ResidualLSTM）。实验结果显示，将ResidualLSTM加入Transformer后，模型性能相比原版下降23.37%（以BLEU分数衡量）。相比之下，仅使用Transformer的模型相比Seq2Seq模型，其BLEU分数提升28.14点。
原文摘要 · Abstract (English)
Sign language translation is one of the important issues in communication between deaf and hearing people, as it expresses words through hand, body, and mouth movements. American Sign Language is one of the sign languages used, one of which is the alphabetic sign. The development of neural machine translation technology is moving towards sign language translation. Transformer became the state-of-the-art in natural language processing. This study compares the Transformer with the Sequence-to-Sequence (Seq2Seq) model in translating sign language to text. In addition, an experiment was conducted by adding Residual Long Short-Term Memory (ResidualLSTM) in the Transformer. The addition of ResidualLSTM to the Transformer reduces the performance of the Transformer model by 23.37% based on the BLEU Score value. In comparison, the Transformer itself increases the BLEU Score value by 28.14 compared to the Seq2Seq model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。