为多模态对话系统设计并验证了用户参与度与亲密度量表。
Development and Validation of Engagement and Rapport Scales for Evaluating User Experience in Multimodal Dialogue Systems
- 基于教育心理学等理论构建量表
- 74名日语学习者对比人机对话体验差异
- 量表可有效区分人机交互质量
本研究旨在开发并验证两个用于评估多模态对话系统在外语学习情境中用户体验质量的参与度与亲密度量表。量表设计基于教育心理学、社会心理学及第二语言习得理论。74名日本英语学习者分别与培训过的真人导师和对话代理完成角色扮演与讨论任务，每轮对话后填写量表。通过克隆巴赫系数分析和一系列验证性因子分析，检验了量表的结构效度与项目可靠性。进一步比较了真人导师与对话代理对话中的参与度与亲密度得分。结果表明，该量表从多个维度成功捕捉到了人机对话体验质量的差异。
原文摘要 · Abstract (English)
This study aimed to develop and validate two scales of engagement and rapport to evaluate the user experience quality with multimodal dialogue systems in the context of foreign language learning. The scales were designed based on theories of engagement in educational psychology, social psychology, and second language acquisition.Seventy-four Japanese learners of English completed roleplay and discussion tasks with trained human tutors and a dialog agent. After each dialogic task was completed, they responded to the scales of engagement and rapport. The validity and reliability of the scales were investigated through two analyses. We first conducted analysis of Cronbach's alpha coefficient and a series of confirmatory factor analyses to test the structural validity of the scales and the reliability of our designed items. We then compared the scores of engagement and rapport between the dialogue with human tutors and the one with a dialogue agent. The results revealed that our scales succeeded in capturing the difference in the dialogue experience quality between the human interlocutors and the dialogue agent from multiple perspectives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。