为听障人士设计触控语音助手，提升智能助理可用性
Deaf and Hard of Hearing Access to Intelligent Personal Assistants: Comparison of Voice-Based Options with an LLM-Powered Touch Interface
- 用大模型驱动触控界面替代语音输入
- 触控与语音在可用性上无显著差异
- 需原生支持听障口音的语音识别
我们研究了听障和重听（DHH）人群使用智能个人助理（IPAs）的可访问性问题，他们虽能发声但常因语音识别系统无法理解多样口音（包括听障口音）而难以使用。通过一台Echo Show，我们在混合方法研究中比较了自然语言通过英语语音输入、Alexa自动语音识别与训练人员转述命令（伪巫师设置）的可用性，以及基于大语言模型（LLM）的触控界面表现。触控界面由一个集成用户历史与智能环境的LLM驱动“任务提示器”引导。定量结果显示，两种语音条件与触控方式在可用性上无显著差异；定性分析显示对各方法的可接受度存在个体差异。最终结论是：必须让智能助理原生支持听障口音的语音识别。
原文摘要 · Abstract (English)
We investigate intelligent personal assistants (IPAs) accessibility for deaf and hard of hearing (DHH) people who can use their voice in everyday communication. The inability of IPAs to understand diverse accents including deaf speech renders them largely inaccessible to non-signing and speaking DHH individuals. Using an Echo Show, we compare the usability of natural language input via spoken English; with Alexa's automatic speech recognition and a Wizard-of-Oz setting with a trained facilitator re-speaking commands against that of a large language model (LLM)-assisted touch interface in a mixed-methods study. The touch method was navigated through an LLM-powered "task prompter," which integrated the user's history and smart environment to suggest contextually-appropriate commands. Quantitative results showed no significant differences across both spoken English conditions vs LLM-assisted touch. Qualitative results showed variability in opinions on the usability of each method. Ultimately, it will be necessary to have robust deaf-accented speech recognized natively by IPAs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。