对比多种姿态估计算法对手语翻译效果的影响，发现新模型显著提升准确率。
Evaluation of Pose Estimation Systems for Sign Language Translation
- 在相同翻译管道下测试不同姿态估计器，控制变量评估性能差异。
- SDPose和Sapiens表现最优，BLEU达11.5，优于常用MediaPipe的10。
- 手部关键点缺失多的模型翻译效果差，且对遮挡鲁棒性弱。
许多手语翻译（SLT）系统采用姿态序列而非原始视频，以降低输入维度、提升可移植性并部分匿名化签署者。姿态估计算法的选择常被视为实现细节，系统普遍采用MediaPipe Holistic或OpenPose等通用工具。本文系统比较了用于姿态型SLT的多种姿态估计算法，涵盖主流基线（MediaPipe Holistic、OpenPose）与新型全身/高容量模型（MMPose WholeBody、OpenPifPaf、AlphaPose、SDPose、Sapiens、SMPLest-X）。通过在RWTH-PHOENIX-Weather 2014数据集上训练统一的SLT管道，仅改变姿态表示，使用BLEU和BLEURT评估下游性能。为理解翻译结果，还利用Signsuisse数据集的高分辨率视频分析时间稳定性、手部关键点缺失及遮挡鲁棒性。结果显示，SDPose和Sapiens表现最佳（BLEU ~11.5），优于常用的MediaPipe基线（BLEU ~10）。在遮挡情况下，Sapiens在15次测试中全部正确（15/15），而OpenPifPaf几乎全失败（1/15），且翻译得分最差。频繁遗漏手部关键点的估计算法对应更低的BLEU/BLEURT分数。代码已开源，便于复现及降低其他研究者使用替代模型的门槛。
原文摘要 · Abstract (English)
Many sign language translation (SLT) systems operate on pose sequences instead of raw video to reduce input dimensionality, improve portability, and partially anonymize signers. The choice of pose estimator is often treated as an implementation detail, with systems defaulting to widely available tools such as MediaPipe Holistic or OpenPose. We present a systematic comparison of pose estimators for pose-based SLT, covering widely used baselines (MediaPipe Holistic, OpenPose) and newer whole-body/high-capacity models (MMPose WholeBody, OpenPifPaf, AlphaPose, SDPose, Sapiens, SMPLest-X). We quantify downstream impact by training a controlled SLT pipeline on RWTH-PHOENIX-Weather 2014 where only the pose representation varies, evaluating with BLEU and BLEURT. To contextualize translation outcomes, we analyze temporal stability, missing hand keypoints, and robustness to occlusion using higher-resolution videos from the Signsuisse dataset. SDPose and Sapiens achieve the best translation performance (BLEU ~11.5), outperforming the common MediaPipe baseline (BLEU ~10). In occlusion cases, Sapiens is correct in all tested instances (15/15), while OpenPifPaf fails in nearly all (1/15) and also yields the weakest translation scores. Estimators that frequently leave out hand keypoints are associated with lower BLEU/BLEURT. We release code that can be used not only to reproduce our experiments, but also considerably lowers the barrier for other researchers to use alternative pose estimators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。