用普通机器人实现手语沟通,让聋人更轻松交流。
Using the Pepper Robot to Support Sign Language Communication
- 用动画和逆运动学在Pepper机器人上实现52个意大利手语动作。
- 单个手势识别正确率高,但完整句子识别率低。
- 适合关注无障碍人机交互的开发者和研究者。
社交机器人在公共和辅助场景中日益普及,但对聋人群体的可用性仍鲜有研究。意大利手语(LIS)是一种包含复杂手动与非手动成分的完整自然语言。让机器人能使用LIS沟通,有助于促进包容性人机交互,尤其在医院、机场或教育场所。本研究探讨商用社交机器人Pepper能否清晰表达LIS手势及短句。在聋人学生及其翻译专家协助下,我们通过手动动画或基于MATLAB的逆运动学求解器,实现了52个LIS手势。12位精通LIS的参与者(含聋人与听人)完成问卷,包含15个视频选择题和2个开放问题。结果表明,大多数孤立手势可被正确识别,但完整句子识别率显著偏低,主要因机器人表达能力有限及时间约束。研究证明,即使是商用机器人如Pepper也能以可理解方式执行部分LIS手势,为更包容的设计提供可能。未来需加强多模态支持(如屏幕提示或表情化身),并让聋人参与设计,提升机器人表现力与可用性。
原文摘要 · Abstract (English)
Social robots are increasingly experimented in public and assistive settings, but their accessibility for Deaf users remains quite underexplored. Italian Sign Language (LIS) is a fully-fledged natural language that relies on complex manual and non-manual components. Enabling robots to communicate using LIS could foster more inclusive human robot interaction, especially in social environments such as hospitals, airports, or educational settings. This study investigates whether a commercial social robot, Pepper, can produce intelligible LIS signs and short signed LIS sentences. With the help of a Deaf student and his interpreter, an expert in LIS, we co-designed and implemented 52 LIS signs on Pepper using either manual animation techniques or a MATLAB based inverse kinematics solver. We conducted a exploratory user study involving 12 participants proficient in LIS, both Deaf and hearing. Participants completed a questionnaire featuring 15 single-choice video-based sign recognition tasks and 2 open-ended questions on short signed sentences. Results shows that the majority of isolated signs were recognized correctly, although full sentence recognition was significantly lower due to Pepper's limited articulation and temporal constraints. Our findings demonstrate that even commercially available social robots like Pepper can perform a subset of LIS signs intelligibly, offering some opportunities for a more inclusive interaction design. Future developments should address multi-modal enhancements (e.g., screen-based support or expressive avatars) and involve Deaf users in participatory design to refine robot expressivity and usability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。