arXiv:2410.15433q-bio.NCcs.CV2024-10ICLR被引 10

用主畸变分析图像表征的局部差异,直观区分不同模型敏感性。

Discriminating image representations with principal distortions

  • 基于费雪信息矩阵量化表征对局部畸变的敏感度。
  • 找出能最大化模型差异的两组主畸变,实现最优区分。
  • 适用于对比神经网络与视觉系统模型的局部感知特性。

图像表征(人工或生物)常以全局几何结构进行比较;然而,具有相似全局结构的表征可能在局部几何上存在显著差异。本文提出一种框架,用于从局部几何角度比较一组图像表征。通过费雪信息矩阵这一标准统计工具,量化表征对局部刺激畸变的敏感性,并以此构建基图附近的局部几何度量。该度量可用于最优区分多组模型,即寻找一对“主畸变”,使模型在该度量下的方差最大化。作为示例,我们用此框架比较早期视觉系统的若干简单模型,识别出一组新颖的图像畸变,使模型可通过视觉直接对比。在第二个例子中,将方法应用于深度神经网络模型,揭示了因架构和训练方式不同而产生的局部几何差异。这些结果表明,该框架可有效探测复杂模型间的局部敏感性差异,并为模型与人类感知的比较提供新思路。

原文摘要 · Abstract (English)

Image representations (artificial or biological) are often compared in terms of their global geometric structure; however, representations with similar global structure can have strikingly different local geometries. Here, we propose a framework for comparing a set of image representations in terms of their local geometries. We quantify the local geometry of a representation using the Fisher information matrix, a standard statistical tool for characterizing the sensitivity to local stimulus distortions, and use this as a substrate for a metric on the local geometry in the vicinity of a base image. This metric may then be used to optimally differentiate a set of models, by finding a pair of "principal distortions" that maximize the variance of the models under this metric. As an example, we use this framework to compare a set of simple models of the early visual system, identifying a novel set of image distortions that allow immediate comparison of the models by visual inspection. In a second example, we apply our method to a set of deep neural network models and reveal differences in the local geometry that arise due to architecture and training types. These examples demonstrate how our framework can be used to probe for informative differences in local sensitivities between complex models, and suggest how it could be used to compare model representations with human perception.

图像表征费雪信息模型比较局部几何

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。