用视频查手语词典,让初学者更高效学习。
Towards an AI-Driven Video-Based American Sign Language Dictionary: Exploring Design and Usage Experience with Learners
- 基于最新手语识别技术,实现视频输入匹配手语符号
- 12名新手使用者在任务中暴露系统延迟与输出不可预测问题
- 揭示隐私与重录需求,指导未来交互设计
学习者查询不熟悉的手语符号时面临挑战,因无法像语音语言一样输入文字。孤立手语识别技术进步使视频型词典成为可能:用户上传视频,系统返回最匹配的手语符号列表。本研究结合以往巫师-奥兹原型的界面设计建议,并采用先进手语识别技术,开发出自动化视频词典。通过观察12名初学者在视频理解与问答任务中的使用情况,我们发现此前未被覆盖的人机交互问题,包括录制与重传符号、输出不可预测性、系统延迟及隐私顾虑。这些发现为视频型手语词典系统的构建与部署提供了实践指导。
原文摘要 · Abstract (English)
Searching for unfamiliar American Sign Language (ASL) signs is challenging for learners because, unlike spoken languages, they cannot type a text-based query to look up an unfamiliar sign. Advances in isolated sign recognition have enabled the creation of video-based dictionaries, allowing users to submit a video and receive a list of the closest matching signs. Previous HCI research using Wizard-of-Oz prototypes has explored interface designs for ASL dictionaries. Building on these studies, we incorporate their design recommendations and leverage state-of-the-art sign-recognition technology to develop an automated video-based dictionary. We also present findings from an observational study with twelve novice ASL learners who used this dictionary during video-comprehension and question-answering tasks. Our results address human-AI interaction challenges not covered in previous WoZ research, including recording and resubmitting signs, unpredictable outputs, system latency, and privacy concerns. These insights offer guidance for designing and deploying video-based ASL dictionary systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。