arXiv:2604.22045cs.CVcs.AI2026-04被引 1

发现图像分类中像素间的协同作用,提升模型解释力。

H-Sets: Hessian-Guided Discovery of Set-Level Feature Interactions in Image Classifiers

论文配图:H-Sets: Hessian-Guided Discovery of Set-Level Feature Interactions in Image Classifiers
图 1 · 摘自论文原文
  • 用海森矩阵检测局部相关像素对,递归合并成语义一致的特征集。
  • 在ImageNet和CUB上,生成的显著图更稀疏且更忠实于真实决策。
  • 适合需要精细解释视觉决策机制的研究者使用。

特征归因方法通过为输入特征分配重要性分数来解释深度神经网络的预测结果。然而,大多数现有方法仅关注单个特征的边际效应,忽略了特征间相互作用——即多个特征共同影响模型输出的情况。这类相互作用在图像分类任务中尤为重要,因为语义意义往往源于像素间的依赖关系而非孤立特征。现有的基于交互作用的方法在图像上要么过于粗糙(如仅基于超像素),要么不满足核心可解释性公理。本文提出H-Sets,一种两阶段框架,用于发现并归因图像分类器中的高阶特征交互。首先,利用输入海森矩阵检测局部相互作用的像素对,并递归合并成语义连贯的特征集;采用Segment Anything (SAM)进行空间分组作为先验,也可替换为其他分割结果。其次,采用IDG-Vis方法为每个特征集赋予权重,该方法是积分方向梯度(Integrated Directional Gradients)的集合扩展,结合像素空间路径的方向梯度,并以哈桑尼红利聚合。虽然海森矩阵引入了额外计算开销,但这一针对性成本带来了更稀疏、更忠实的显著性图。在VGG、ResNet、DenseNet和MobileNet模型上,对ImageNet和CUB数据集的评估表明,与现有方法相比,H-Sets生成的显著性图更具可解释性和忠实度。

原文摘要 · Abstract (English)

Feature attribution methods explain the predictions of deep neural networks by assigning importance scores to individual input features. However, most existing methods focus solely on marginal effects, overlooking feature interactions, where groups of features jointly influence model output. Such interactions are especially important in image classification tasks, where semantic meaning often arises from pixel interdependencies rather than isolated features. Existing interaction-based methods for images are either coarse (e.g., superpixel-only) or, fail to satisfy core interpretability axioms. In this work, we introduce H-Sets, a novel two-stage framework for discovering and attributing higher-order feature interactions in image classifiers. First, we detect locally interacting pairs via input Hessians and recursively merge them into semantically coherent sets; segmentation from Segment Anything (SAM) is used as a spatial grouping prior but can be replaced by other segmentations. Second, we attribute each set with IDG-Vis, a set-level extension of Integrated Directional Gradients that integrates directional gradients along pixel-space paths and aggregates them with Harsanyi dividends. While Hessians introduce additional compute at the detection stage, this targeted cost consistently yields saliency maps that are sparser and more faithful. Evaluations across VGG, ResNet, DenseNet and MobileNet models on ImageNet and CUB datasets show that H-Sets generate more interpretable and faithful saliency maps compared to existing methods.

模型解释特征交互海森矩阵显著图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。