用眼动数据自动判断人解决视觉任务的熟练度,5秒内准确率达75.1%。
AutoSIGHT: Automatic Eye Tracking-based System for Immediate Grading of Human experTise
- 基于眼动特征构建集成模型,实时分析人类视觉任务表现
- 5秒评估窗口下AUROC达0.751,30秒时提升至0.8306
- 适用于动态人机协作场景,适合研究人机能力匹配的学者
能否让机器仅通过眼动数据自动评估人类在视觉任务中的熟练程度?本文提出AutoSIGHT系统,通过分析受试者完成虹膜活体检测(iris PAD)任务时的眼动特征,实现对专家与非专家的分类。实验结果表明,在仅5秒的短评估窗口下,该系统在跨被试训练-测试设置中平均AUROC达到0.751,证明其有效性;当评估时间延长至30秒时,AUROC提升至0.8306,说明模型能更充分地利用信息,但决策略有延迟。本研究为如何在人机协作系统中动态融合人机能力提供了新思路,尤其适用于专家能力随任务变化或需快速替换的场景。本文还公开了来自6名专家和53名非专家完成iris PAD任务的眼动数据集。
原文摘要 · Abstract (English)
Can we teach machines to assess the expertise of humans solving visual tasks automatically based on eye tracking features? This paper proposes AutoSIGHT, Automatic System for Immediate Grading of Human experTise, that classifies expert and non-expert performers, and builds upon an ensemble of features extracted from eye tracking data while the performers were solving a visual task. Results on the task of iris Presentation Attack Detection (PAD) used for this study show that with a small evaluation window of just 5 seconds, AutoSIGHT achieves an average average Area Under the ROC curve performance of 0.751 in subject-disjoint train-test regime, indicating that such detection is viable. Furthermore, when a larger evaluation window of up to 30 seconds is available, the Area Under the ROC curve (AUROC) increases to 0.8306, indicating the model is effectively leveraging more information at a cost of slightly delayed decisions. This work opens new areas of research on how to incorporate the automatic weighing of human and machine expertise into human-AI pairing setups, which need to react dynamically to nonstationary expertise distribution between the human and AI players (e.g. when the experts need to be replaced, or the task at hand changes rapidly). Along with this paper, we offer the eye tracking data used in this study collected from 6 experts and 53 non-experts solving iris PAD visual task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。