发现并定位视觉模型偏见的隐藏空间模式,助力精准审计模型偏差。
Discovery and Spatial Characterisation of Multiple Shortcut Groups for Auditing Vision Model Bias

- 用聚类与矩阵分解方法从图像中提取重复出现的偏见区域模式。
- 在多个数据集上识别出不同子群体的偏见模式,误差率差异显著。
- 适合研究模型公平性、可解释性或用于调试深度学习模型的工程师。
在存在伪相关性的数据集上训练的深度学习模型虽能取得高平均准确率,但可能依赖于无法泛化的捷径特征。现有方法通过可解释性技术生成贡献图来分析这些特征的空间分布,但通常对全数据集聚合结果,可能掩盖仅在部分图像中出现的重复性空间模式。本文提出将每张图像的捷径与任务贡献图进行聚类(采用K-means和非负矩阵分解),识别出反复出现的空间模式,并通过贡献图和代表性样本可视化。在CelebA、CheXpert、Waterbirds、Camelyon17和ISIC2019数据集上,使用ResNet和ViT模型验证,发现不同子群体存在共享与独特的捷径与任务贡献空间模式,且误差率各异,支持对高错误率图像子集的针对性分析。通过输入遮蔽和测试时干预实验表明,抑制任务贡献区域会显著降低分类性能;进一步提出捷径抑制与任务增强相结合的特征干预策略,有效降低性能差异。
原文摘要 · Abstract (English)
Deep learning models trained on datasets with spurious correlations can achieve high average accuracy whilst relying on shortcut features that do not generalise out of distribution. Whilst out-of-distribution testing highlights subgroup performance disparities arising from shortcut learning, it does not localise the regions within images that are associated with it. Existing research mostly uses attribution maps from interpretability methods to understand the spatial nature of spurious correlations. For example, conditional alignment methods separate task-relevant evidence from evidence tied to spurious correlations by comparing attribution maps from a task model, a sensitive attribute model, and a bias-reduced reference model. This yields shortcut-aligned and task-aligned contribution maps for each image. However, existing methods aggregate these maps across the dataset, potentially masking recurring spatial shortcut patterns that occur only in subsets of images. We address this limitation by grouping per-image shortcut and task contribution maps into recurring spatial patterns using K-means and non-negative matrix factorisation, and visualising the resulting shortcut groups through contribution maps and representative examples. Across CelebA, CheXpert, Waterbirds, Camelyon17, and ISIC2019, and across ResNet and ViT models, the discovered shortcut groups reveal both shared and distinct spatial patterns of shortcut and task contribution, with varying subgroup composition and error rates, enabling targeted inspection of image subsets with higher error rates. We perform input occlusion and internal test-time interventions to show that masking or suppressing task contribution regions substantially degrades the model classification performance and propose a combined shortcut suppression and task amplification feature intervention approach which generally reduces performance disparities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。