让神经网络像人一样快速理解规则并灵活应变。
Sparks of cognitive flexibility: self-guided context inference for flexible stimulus-response mapping by attentional routing
- 用可调上下文状态引导注意力,动态聚焦关键特征。
- 仅需少量样本就能推断隐藏规则,且不遗忘旧知识。
- 适合需要快速适应新规则的复杂视觉任务研究者。
灵活认知需要快速发现隐藏规则以适应刺激-反应映射。标准神经网络在需快速、上下文驱动重映射的任务中表现不佳。近期Hummos(2023)提出快慢学习算法缓解此问题,但其在复杂图像可计算任务中的可扩展性尚不明确。本文提出威斯康星神经网络(WiNN),将快慢学习拓展至需灵活规则行为的图像任务。WiNN采用预训练卷积神经网络处理视觉输入,并引入可调节的“上下文状态”以引导注意力至相关特征。若响应错误,先迭代更新上下文状态以重新聚焦任务相关线索,再对注意力与读出层进行最小参数调整。该策略保留感知与注意力网络的可泛化表示,降低灾难性遗忘。我们在威斯康星卡片分类任务的图像扩展版本上评估了WiNN,结果显示:(i)WiNN自主推断底层规则;(ii)所需样本数远少于依赖大规模参数更新的对照模型;(iii)仅通过上下文状态调整即可实现基于上下文的规则推理,结合慢速参数更新进一步提升性能;(iv)仅通过上下文状态更新即可泛化至未见的组合规则。通过融合快速上下文推断与定向注意力引导,WiNN实现了认知灵活性的“火花”。该方法为保持知识的同时快速适应复杂规则任务提供了新路径。
原文摘要 · Abstract (English)
Flexible cognition demands discovering hidden rules to quickly adapt stimulus-response mappings. Standard neural networks struggle in such tasks requiring rapid, context-driven remapping. Recently, Hummos (2023) introduced a fast-and-slow learning algorithm to mitigate this shortcoming, but its scalability to complex, image-computable tasks was unclear. Here, we propose the Wisconsin Neural Network (WiNN), which extends Hummos' fast-and-slow learning to image-computable tasks demanding flexible rule-based behavior. WiNN employs a pretrained convolutional neural network for vision, coupled with an adjustable "context state" that guides attention to relevant features. If WiNN produces an incorrect response, it first iteratively updates its context state to refocus attention on task-relevant cues, then performs minimal parameter updates to attention and readout layers. This strategy preserves generalizable representations in the sensory and attention networks, reducing catastrophic forgetting. We evaluate WiNN on an image-based extension of the Wisconsin Card Sorting Task, revealing several markers of cognitive flexibility: (i) WiNN autonomously infers underlying rules, (ii) requires fewer examples to do so than control models reliant on large-scale parameter updates, (iii) can perform context-based rule inference solely via context-state adjustments-further enhanced by slow updates of attention and readout parameters, and (iv) generalizes to unseen compositional rules through context-state updates alone. By blending fast context inference with targeted attentional guidance, WiNN achieves "sparks" of flexibility. This approach offers a path toward context-sensitive models that retain knowledge while rapidly adapting to complex, rule-based tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。