通过眼动追踪与多模态语义分析，研究未来场景理解中的认知负担。
FutureVision: A methodology for the investigation of future cognition
- 结合眼动实验与多模态语义标注，分析未来场景理解过程。
- 远未来和悲观场景引发更长注视和不规则眼跳，认知负荷更高。
- 适合认知科学、人机交互及未来学研究者参考。
本文提出一种结合多模态语义分析与眼动追踪实验的框架，用于研究理解未来情境沟通时的认知努力。为验证该方法，我们开展一项试点研究，利用便携式眼动仪记录参与者在评估虚构广告中未来场景的正负性与反事实性时的眼动轨迹，并同步记录其向对话伙伴描述刺激内容的过程。眼动模式与基于框架语义标注的语义表示（涵盖语言与视觉模态）进行联合分析。初步结果显示，远未来和悲观性场景伴随更长的注视时间与更不规则的扫视，支持了未来情境理解基础空间断裂会增加理解者认知负荷的假设。
原文摘要 · Abstract (English)
This paper presents a methodology combining multimodal semantic analysis with an eye-tracking experimental protocol to investigate the cognitive effort involved in understanding the communication of future scenarios. To demonstrate the methodology, we conduct a pilot study examining how visual fixation patterns vary during the evaluation of valence and counterfactuality in fictional ad pieces describing futuristic scenarios, using a portable eye tracker. Participants eye movements are recorded while evaluating the stimuli and describing them to a conversation partner. Gaze patterns are analyzed alongside semantic representations of the stimuli and participants descriptions, constructed from a frame semantic annotation of both linguistic and visual modalities. Preliminary results show that far-future and pessimistic scenarios are associated with longer fixations and more erratic saccades, supporting the hypothesis that fractures in the base spaces underlying the interpretation of future scenarios increase cognitive load for comprehenders.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。