用遮蔽归因法对比人和模型的视觉策略,验证解释有效性。
MAPS: Masked Attribution-based Probing of Strategies- A computational framework to align human and model explanations
- 将归因图转为解释遮蔽图,测试最小信息下的识别准确率
- 在模拟与真实实验中,成功还原模型间策略相似性
- 只需少量行为数据,适合快速验证模型解释与生物视觉的一致性
人类核心物体识别依赖于对视觉信息的选择性使用,但引导选择的策略难以直接测量。我们提出MAPS(基于遮蔽归因的策略探测框架),一种经过行为验证的计算工具,用于检验人工神经网络(ANN)的解释是否也能说明人类视觉。MAPS将归因图转化为解释遮蔽图像(EMI),并逐图像比较人类在有限像素预算下对这些最小化图像的识别准确率与对完整刺激的准确率。该方法为评估和选择不同ANN可解释性方法提供了原则性依据。在模拟中,基于EMI的行为相似性可靠地恢复了由归因图计算出的真实相似性,从而确定哪种解释方法最能捕捉模型策略。应用于人类和猕猴时,MAPS识别出与生物视觉最接近的模型-解释组合,其行为有效性媲美泡泡掩码(Bubble masks),但所需行为试验次数少得多。由于仅需模型归因和原始图像上的少量行为数据,MAPS避免了繁琐的心理物理实验,提供了一种可扩展的工具,统一评估解释、人类行为、神经活动和模型决策。
原文摘要 · Abstract (English)
Human core object recognition depends on the selective use of visual information, but the strategies guiding these choices are difficult to measure directly. We present MAPS (Masked Attribution-based Probing of Strategies), a behaviorally validated computational tool that tests whether explanations derived from artificial neural networks (ANNs) can also explain human vision. MAPS converts attribution maps into explanation-masked images (EMIs) and compares image-by-image human accuracies on these minimal images with limited pixel budgets with accuracies on the full stimuli. MAPS provides a principled way to evaluate and choose among competing ANN interpretability methods. In silico, EMI-based behavioral similarity between models reliably recovers the ground-truth similarity computed from their attribution maps, establishing which explanation methods best capture the model's strategy. When applied to humans and macaques, MAPS identifies ANN-explanation combinations whose explanations align most closely with biological vision, achieving the behavioral validity of Bubble masks while requiring far fewer behavioral trials. Because it needs only access to model attributions and a modest set of behavioral data on the original images, MAPS avoids exhaustive psychophysics while offering a scalable tool for adjudicating explanations and linking human behavior, neural activity, and model decisions under a common standard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。