提出跨领域通用认知负荷评估新方法,提升真实场景下的模型泛化能力。
REVELIO -- Universal Multimodal Task Load Estimation for Cross-Domain Generalization
- 基于n-back测试构建多模态真实游戏数据集,融合客观表现与主观评分
- 多模态模型在跨域任务中表现优于单模态,但迁移性能显著下降
- 为自适应系统设计提供可复用基准,适合人机交互研究者参考
任务负荷检测对优化人类表现至关重要,但现有模型常局限于特定实验场景。本文通过扩展经典认知负荷检测基准,引入基于n-back测试的真实游戏应用,构建新的多模态数据集。任务负荷标注综合客观性能、主观NASA-TLX评分及任务设计,形成全面评估框架。采用xLSTM、ConvNeXt、Transformer等先进端到端模型,在多种模态和应用领域上系统训练与评估其预测性能与跨域泛化能力。结果表明,多模态方法始终优于单模态基线,不同模态与模型架构的影响随应用子集而异。重要的是,单一领域训练的模型在新应用场景中性能明显下降,凸显通用认知负荷估计仍面临挑战。研究为开发更具泛化性的负荷检测系统提供了可靠基准与实践指导,推动人机交互与自适应系统的发展。
原文摘要 · Abstract (English)
Task load detection is essential for optimizing human performance across diverse applications, yet current models often lack generalizability beyond narrow experimental domains. While prior research has focused on individual tasks and limited modalities, there remains a gap in evaluating model robustness and transferability in real-world scenarios. This paper addresses these limitations by introducing a new multimodal dataset that extends established cognitive load detection benchmarks with a real-world gaming application, using the $n$-back test as a scientific foundation. Task load annotations are derived from objective performance, subjective NASA-TLX ratings, and task-level design, enabling a comprehensive evaluation framework. State-of-the-art end-to-end model, including xLSTM, ConvNeXt, and Transformer architectures are systematically trained and evaluated on multiple modalities and application domains to assess their predictive performance and cross-domain generalization. Results demonstrate that multimodal approaches consistently outperform unimodal baselines, with specific modalities and model architectures showing varying impact depending on the application subset. Importantly, models trained on one domain exhibit reduced performance when transferred to novel applications, underscoring remaining challenges for universal cognitive load estimation. These findings provide robust baselines and actionable insights for developing more generalizable cognitive load detection systems, advancing both research and practical implementation in human-computer interaction and adaptive systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。