研究自然对话中手语如何随语境变化,揭示互动对表达方式的影响。
How Pragmatics Shape Articulation: A Computational Case Study in STEM ASL Discourse
- 通过动作捕捉分析对话与独白手语的差异,分离出互动影响与个体省力行为。
- 对话中的手语平均比孤立手语短24.6%至44.6%,且该缩短在独白中不显著。
- 适合关注手语动态性、教育场景建模及交互式手语技术的研究者。
当前多数手语模型基于口译或孤立词汇数据训练,忽略了真实对话中的多样性。人类交流会根据上下文和对话对象动态调整空间时间特征与表达风格。尤其在教育场景中,教师与学生会共同使用新词汇。为此,我们收集了美国手语(ASL)STEM(科学、技术、工程与数学)对话的动作捕捉数据集,实现对双人互动手语、单人讲授手语与口译文本的定量比较。利用连续运动学特征,我们分离出对话特有的同步效应与个体努力降低,并发现重复提及的STEM术语存在显著的空间时间变化。平均而言,对话手语持续时间比孤立手语缩短24.6%至44.6%,且这种减少在独白情境中不明显。最后,我们评估了手语嵌入模型在识别STEM手语及量化参与者同步程度上的表现。本研究连接语言学分析与计算建模,揭示语用如何塑造手语表达及其在手语技术中的表征。
原文摘要 · Abstract (English)
Most state-of-the-art sign language models are trained on interpreter or isolated vocabulary data, which overlooks the variability that characterizes natural dialogue. However, human communication dynamically adapts to contexts and interlocutors through spatiotemporal changes and articulation style. This specifically manifests itself in educational settings, where novel vocabularies are used by teachers, and students. To address this gap, we collect a motion capture dataset of American Sign Language (ASL) STEM (Science, Technology, Engineering, and Mathematics) dialogue that enables quantitative comparison between dyadic interactive signing, solo signed lecture, and interpreted articles. Using continuous kinematic features, we disentangle dialogue-specific entrainment from individual effort reduction and show spatiotemporal changes across repeated mentions of STEM terms. On average, dialogue signs are 24.6%-44.6% shorter in duration than the isolated signs, and show significant reductions absent in monologue contexts. Finally, we evaluate sign embedding models on their ability to recognize STEM signs and approximate how entrained the participants become over time. Our study bridges linguistic analysis and computational modeling to understand how pragmatics shape sign articulation and its representation in sign language technologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。