ELMI让聋哑歌手轻松将歌词转为手语，支持实时编辑与AI对话。
ELMI: Interactive and Intelligent Sign Language Translation of Lyrics for Song Signing
- 用户可逐行编辑手语词，实时同步歌词与音乐视频。
- 13位歌手参与测试，普遍提升翻译信心与独立性。
- 内置大模型聊天功能，辅助理解、表达和节奏把控。
聋哑人与听力正常者在短视频平台上的歌曲手语表演日益普遍，但歌词到手语的翻译仍繁琐且难获取。我们的前期研究揭示了手语译者面临的语义、语法、表现力和节奏等挑战。本文提出ELMI，一款面向歌曲手语翻译的易用工具，支持用户逐行编辑手语词，并实时同步歌词与音乐视频片段。用户还可通过大语言模型驱动的AI聊天功能，讨论意义、手语词选择、情感表达与时间节奏。通过与13位手语歌手的探索性研究，我们发现参与者成功采用ELMI完成手语创作，全程积极互动。他们报告翻译信心与自主性显著提升，认为ELMI具鼓励性、建设性和启发性。本文还探讨了面向无障碍与文化敏感性的手语翻译工具的研究与设计启示。
原文摘要 · Abstract (English)
d/Deaf and hearing song-signers have become prevalent across video-sharing platforms, but translating songs into sign language remains cumbersome and inaccessible. Our formative study revealed the challenges song-signers face, including semantic, syntactic, expressive, and rhythmic considerations in translations. We present ELMI, an accessible song-signing tool that assists in translating lyrics into sign language. ELMI enables users to edit glosses line-by-line, with real-time synced lyric and music video snippets. Users can also chat with a large language model-driven AI to discuss meaning, glossing, emoting, and timing. Through an exploratory study with 13 song-signers, we examined how ELMI facilitates their workflows and how song-signers leverage and receive an LLM-driven chat for translation. Participants successfully adopted ELMI to song-signing, with active discussions throughout. They also reported improved confidence and independence in their translations, finding ELMI encouraging, constructive, and informative. We discuss research and design implications for accessible and culturally sensitive song-signing translation tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。