用人体姿态数据直接检测手语序列中是否包含目标手势，高效准确。
Pose-Based Sign Language Spotting via an End-to-End Encoder Architecture
- 基于关键点的端到端编码器，跳过中间识别步骤。
- 在WSLP 2025数据集上达61.88%准确率和60.00% F1值。
- 适合需要低延迟、抗干扰的手语检索场景。
自动手语识别（ASLR）已成为弥合聋人与听人社区鸿沟的重要领域。然而，从连续手语序列中检索特定手势或检测其存在性的问题仍鲜有研究。本文将此任务定义为手语定位（Sign Language Spotting）。我们提出首个面向手语检索的端到端模型，直接在手语视频提取的姿态关键点上进行操作，判断查询手势是否存在于目标序列中。不同于依赖中间词素识别或文本匹配的传统方法，本方案采用仅编码器结构，配合二分类头实现判断。通过聚焦姿态表示而非原始RGB帧，显著降低计算开销并减少视觉噪声影响。我们在WSLP 2025共享任务的“词存在预测”数据集上进行评估，取得61.88%准确率和60.00% F1分数。结果验证了基于姿态的框架在手语定位中的有效性，为未来自动手语检索与验证研究奠定坚实基础。代码已公开于https://github.com/EbimoJohnny/Pose-Based-Sign-Language-Spotting。
原文摘要 · Abstract (English)
Automatic Sign Language Recognition (ASLR) has emerged as a vital field for bridging the gap between deaf and hearing communities. However, the problem of sign-to-sign retrieval or detecting a specific sign within a sequence of continuous signs remains largely unexplored. We define this novel task as Sign Language Spotting. In this paper, we present a first step toward sign language retrieval by addressing the challenge of detecting the presence or absence of a query sign video within a sentence-level gloss or sign video. Unlike conventional approaches that rely on intermediate gloss recognition or text-based matching, we propose an end-to-end model that directly operates on pose keypoints extracted from sign videos. Our architecture employs an encoder-only backbone with a binary classification head to determine whether the query sign appears within the target sequence. By focusing on pose representations instead of raw RGB frames, our method significantly reduces computational cost and mitigates visual noise. We evaluate our approach on the Word Presence Prediction dataset from the WSLP 2025 shared task, achieving 61.88\% accuracy and 60.00\% F1-score. These results demonstrate the effectiveness of our pose-based framework for Sign Language Spotting, establishing a strong foundation for future research in automatic sign language retrieval and verification. Code is available at https://github.com/EbimoJohnny/Pose-Based-Sign-Language-Spotting
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。