arXiv:2501.11161cs.LG2025-01

用信息论度量人类在变化任务中如何关注特征,比传统奖励误差模型更准确。

Modeling Attention during Dimensional Shifts with Counterfactual and Delayed Feedback

  • 基于历史经验计算信息论度量来模拟注意力
  • 在维度转移和延迟/反事实反馈下表现最优
  • 适合研究人类决策与注意力机制的学者

注意力可用于在上下文带宽任务中指导选择,即使面对此前未经历过的上下文特征。例如在维度转移场景中,新增特征值,特征与结果的关系可能静态或动态变化。现有研究多聚焦即时反馈下的注意力机制,而对延迟反馈或反事实反馈的研究较少。尽管已有方法基于奖励预测误差(RPE)成功建模人类注意力,但近期研究质疑其在更一般注意力机制中的适用性。替代模型提出可使用信息论指标建模注意力,适用于新刺激。本文比较两种模型:一种基于历史经验的信息论度量,另一种基于迭代更新的奖励预测误差。通过在包含同维与跨维转移、即时、延迟及反事实反馈的上下文带宽任务中进行模拟,发现信息论度量能更好解释人类行为。结果表明,信息论指标可能比RPE更适合预测人类在复杂决策中的注意力,但仍需更多人类行为研究验证。

原文摘要 · Abstract (English)

Attention can be used to inform choice selection in contextual bandit tasks even when context features have not been previously experienced. One example of this is in dimensional shifts, where additional feature values are introduced and the relationship between features and outcomes can either be static or variable. Attentional mechanisms have been extensively studied in contextual bandit tasks where the feedback of choices is provided immediately, but less research has been done on tasks where feedback is delayed or in counterfactual feedback cases. Some methods have successfully modeled human attention with immediate feedback based on reward prediction errors (RPEs), though recent research raises questions of the applicability of RPEs onto more general attentional mechanisms. Alternative models suggest that information theoretic metrics can be used to model human attention, with broader applications to novel stimuli. In this paper, we compare two different methods for modeling how humans attend to specific features of decision making tasks, one that is based on calculating an information theoretic metric using a memory of past experiences, and another that is based on iteratively updating attention from reward prediction errors. We compare these models using simulations in a contextual bandit task with both intradimensional and extradimensional domain shifts, as well as immediate, delayed, and counterfactual feedback. We find that calculating an information theoretic metric over a history of experiences is best able to account for human-like behavior in tasks that shift dimensions and alter feedback presentation. These results indicate that information theoretic metrics of attentional mechanisms may be better suited than RPEs to predict human attention in decision making, though further studies of human behavior are necessary to support these results.

注意力机制决策模型信息论带宽任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。