VR中语音与视角不一致,不影响短期记忆表现。
Audiovisual angle and voice incongruence do not affect audiovisual verbal short-term memory in virtual reality
- 在虚拟现实与电脑屏上对比测试语音-视角错配对记忆的影响。
- 无论显示设备或错配类型,正确回忆率均无显著差异。
- 适合研究视听整合与沉浸式认知的学者参考。
虚拟现实(VR)环境常被用于听觉与认知研究,以模拟真实场景,可能优于传统计算机屏幕。本研究考察了头戴式显示器(HMD)与传统电脑屏幕在音频视觉处理上的影响,重点关注其对认知任务中视听加工的作用。通过两个实验,分别在两种显示设备下研究角度(实验1)和声音(实验2)的视听不一致对序列回忆的影响。采用音频视觉言语序列回忆(avVSR)任务,由具身对话代理(ECA)呈现目标数字序列。尽管主观评估显示HMD条件下沉浸感更强,但显示设备对正确回忆比例无显著影响。在电脑屏上极端角度不一致时,回忆率略有上升,可能因注意力提升,但效果微弱且无实际意义。反应时间在两种设备及所有不一致条件下均未受影响。结果表明,avVSR任务在所研究条件下对视角与声音不一致具有鲁棒性,无论显示设备如何。该研究引入了适用于VR的avVSR任务,增进了对视听整合的理解。
原文摘要 · Abstract (English)
Virtual reality (VR) environments are frequently used in auditory and cognitive research to imitate real-life scenarios, presumably enhancing state-of-the-art approaches with traditional computer screens. However, the effects of different display technologies on audiovisual processing remain underexplored. This study investigated how VR displayed with an head-mounted display (HMD) affects serial recall performance compared to traditional computer monitors, focusing on their effects on audiovisual processing in cognitive tasks. For that matter, we conducted two experiments with both an HMD and a computer monitor as display devices and two types of audiovisual incongruences: angle (Exp. 1) and voice (Exp. 2) incongruence. To quantify cognitive performance an audiovisual verbal serial recall (avVSR) task was developed where an embodied conversational agent (ECA) was animated to speak the target digit sequence. Even though subjective evaluations showed a higher sense of presence in the HMD condition, we found no effect of the display device on the proportion of correctly recalled digits. For the extreme conditions of angle incongruence in the computer monitor presentation the proportion of correctly recalled digits increased marginally, presumably due to raised attention, but the effect is likely too small to be meaningful. Response times were not affected by incongruences in either display device across both experiments. These findings suggest that the avVSR task is robust against angular and voice audiovisual incongruences, irrespective of the display device, at least for the conditions studied here. Hence, the study introduces the avVSR task in VR and contributes to the understanding of audiovisual integration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。