提出高效算法PhaseWin,用线性计算量实现高精度视觉归因。
PhaseWin: An Efficient Search Algorithm for Faithful Visual Attribution

- 分阶段窗口搜索替代逐次重评分,大幅降低计算开销。
- 实验表明仅需约n次前向传播,即达最优归因效果。
- 适合需要快速、精准解释视觉模型决策的场景。
视觉归因是解释现代视觉与视觉-语言模型决策的关键工具,旨在揭示模型判断如何依赖输入图像的局部区域。给定一个被划分为n个区域的图像,忠实归因可建模为有序子集搜索问题:逐步插入选中的区域应尽可能早地恢复目标模型响应。穷举搜索需指数级代价,而广泛使用的贪心搜索仍需O(n²)次模型评估,因每一步需重新评分所有剩余候选。本文提出PhaseWin,一种高效的忠实视觉归因子集搜索算法。它将贪心选择重构为分阶段窗口搜索:交替进行全局候选筛选、自适应剪枝与局部窗口精炼,同时保持贪心搜索的区域排序特性。在单调证据累积条件下分析表明,于特征层级结构假设下,该方法可实现可控的线性评估复杂度,并具备近似贪心的忠实性保障。在图像分类、目标检测、视觉定位和图像描述任务上的大量实验显示,相较其他归因方法,PhaseWin以最少的前向传播次数达到最高忠实度,实证验证了从O(n²)到O(n)的理论缩减。代码已开源:https://github.com/Qihuai27/phasewin-va。
原文摘要 · Abstract (English)
Visual attribution is a fundamental tool for interpreting modern vision and vision-language models, particularly when their decisions must be inspected, diagnosed, or audited. Its goal is to explain how a model's decision depends on local regions of the visual input, typically by assigning an importance ordering over candidate image regions. Given an image partitioned into $n$ regions, faithful attribution can be cast as an ordered subset-search problem, in which progressively inserting the selected regions should recover the target model response as early as possible. Exhaustive search over region subsets incurs exponential cost, while the widely used greedy search still requires a quadratic number of model evaluations, because every selection step rescores all remaining candidates. We propose PhaseWin, an efficient subset-search algorithm for faithful visual attribution. PhaseWin reorganizes greedy region selection into a phased window-search procedure: rather than re-evaluating the full candidate set at every step, it alternates between global candidate screening, adaptive pruning, and localized window refinement, while preserving the essential region-ranking behavior of greedy search. We analyze PhaseWin under monotone evidence-accumulation conditions and show that, under feature-level structural assumptions, it attains controllable linear evaluation complexity together with near-greedy faithfulness guarantees. Extensive experiments on image classification, object detection, visual grounding, and image captioning show that, among all compared attribution methods, PhaseWin reaches high faithfulness with the fewest forward passes, empirically realizing the predicted reduction from $O(n^2)$ to $O(n)$. The code is available at https://github.com/Qihuai27/phasewin-va.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。