用眼动追踪捕捉用户注意力，让AI更懂你的任务意图。
Eye Gaze as a Signal for Conveying User Attention in Contextual AI Systems
- 通过可穿戴眼动设备捕捉用户注视轨迹，作为隐式注意力信号。
- 实验表明眼动数据能显著提升视觉语言模型对上下文的理解能力。
- 适合开发更自然交互的智能助手、人机协作系统的研究者参考。
先进的多模态人工智能代理现已能够与用户协同解决现实世界中的挑战。然而，这些新兴的上下文感知AI系统仍依赖用户与系统之间的显式沟通渠道。我们假设，若能通过隐式方式传递用户的兴趣与意图，将降低交互摩擦并提升用户体验。本文探索了可穿戴眼动追踪技术在传达用户注意力方面的潜力。我们测量了有效映射注视轨迹至物理对象所需的眼动信号质量，并开展实验，将视觉扫描路径历史作为额外上下文提供给视觉语言模型进行查询。结果表明，眼动追踪作为一种用户注意力信号具有高价值，能有效传达用户当前任务和兴趣的关键上下文，从而提升上下文感知AI代理的理解能力。
原文摘要 · Abstract (English)
Advanced multimodal AI agents can now collaborate with users to solve challenges in the world. Yet, these emerging contextual AI systems rely on explicit communication channels between the user and system. We hypothesize that implicit communication of the user's interests and intent would reduce friction and improve user experience when collaborating with AI agents. In this work, we explore the potential of wearable eye tracking to convey signals about user attention. We measure the eye tracking signal quality requirements to effectively map gaze traces to physical objects, then conduct experiments that provide visual scanpath history as additional context when querying vision language models. Our results show that eye tracking provides high value as a user attention signal and can convey important context about the user's current task and interests, improving understanding of contextual AI agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。