轻量级手语翻译系统，帧率降低可显著提升效率。
Towards Compact Sign Language Translation: Frame Rate and Model Size Trade-offs
- 用骨骼提取+线性投影替代大模型，仅77M参数
- 12帧/秒时计算量降75%，BLEU-4仅下降0.53
- 适合移动端部署，无需复杂结构
手语翻译（SLT）将手语视频转化为口语文本，促进聋听群体沟通。现有无词典方法依赖大型编码器-解码器模型，难以部署。本文提出一个仅7700万参数的紧凑系统，结合MMPose骨骼提取与T5-small的单线性投影。通过调整输入帧率，揭示实际效率权衡：在How2Sign数据集上，12帧/秒时序列长度减半，编码器自注意力计算复杂度降低75%，仅导致BLEU-4下降0.53（24帧/秒为10.06，12帧/秒为9.53）。系统规模约为先前T5-base方案的三分之一，证明轻量架构在不使用分层编码器或大规模模型的前提下仍具竞争力。
原文摘要 · Abstract (English)
Sign Language Translation (SLT) converts sign language videos into spoken-language text, bridging communication between Deaf and hearing communities. Current gloss-free approaches rely on large encoder-decoder models, limiting deployment. We propose a compact 77M-parameter pipeline that couples MMPose skeletal pose extraction with a single linear projection into T5-small. By varying the input frame rate, we expose a practical efficiency trade-off: at 12 fps the model halves its sequence length, achieving a 75% reduction in encoder quadratic self-attention computational complexity while incurring only a modest BLEU-4 drop (9.53 vs. 10.06 at 24 fps on How2Sign). Our system is roughly 3x smaller than prior T5-base systems, demonstrating that a lightweight architecture can remain competitive without hierarchical encoders or large-scale models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。