arXiv:2605.07086cs.CVcs.LG2026-05

提出双轴视角,区分通道任务相关性与本地可替代性。

Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Information

论文配图:Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Information
图 1 · 摘自论文原文
  • 用两个维度分离通道重要性:任务相关性与同层替代能力。
  • 本地可替代性比任务相关性更能预测剪枝效果,尤其在固定算力下。
  • 实验证明同伴支持能提升剪枝可靠性,适用于多种网络结构。

视觉网络中通道重要性通常由单一评分概括,但该方式掩盖了两个关键问题:通道与任务的相关程度,以及当通道被移除时,同层其他通道能否替代其功能(即本地可替代性)。本文提出双轴视角,分别衡量输入捕获能力与同伴重叠(本地轴),以及任务信息与目标过量信息(目标轴)。在CIFAR-100上训练的ResNet-18、VGG-16和MobileNetV2中,两轴弱相关,导致不同通道分组,并在训练初期迅速分离,尽管随机初始化时高度耦合。高斯线性分析表明残差梯度方向可解释此分离机制。消融与同伴替换实验显示,同伴支持可进一步提升可移除性判断精度。在固定FLOPs匹配剪枝协议下,本地轴指标对可移除性的预测优于目标轴,在CIFAR-10、Tiny-ImageNet、ImageNet-100及ConvNeXt-T/ImageNet-100测试中均保持一致趋势。结果表明,不应以统一排名评价剪枝分数,本地可替代性是更可靠的剪枝依据,而范数基线在VGG-16等结构中仍具竞争力。

原文摘要 · Abstract (English)

Channel importance in vision networks is usually summarized by a single score. That summary hides two different questions: how much a channel is related to the task, and whether its function can be supplied by same-layer peers when the channel is removed. We call the second property local replaceability. We introduce a two-axis view that separates these questions. The local axis measures input capture and peer overlap, while the target axis measures task information and target-excess information. Across ResNet-18, VGG-16, and MobileNetV2 trained on CIFAR-100, the two axes are weakly aligned, induce different channel groupings, and separate rapidly during training despite being strongly coupled at random initialization. A Gaussian linear analysis accounts for how this separation can arise through residualized gradient directions, and lesion plus peer-replacement experiments show that peer support refines removability beyond input capture and task relevance alone. Under the fixed FLOPs-matched pruning protocol, local-axis metrics are more reliable predictors of removability than target-axis metrics across the three CIFAR-100 backbones, with the same direction preserved in stress tests on CIFAR-10, Tiny-ImageNet, ImageNet-100, and a ConvNeXt-T/ImageNet-100 pilot. These findings identify an axis-level distinction rather than a universal ranking of pruning scores: local replaceability is a more reliable guide to removability than target relevance, while norm-based baselines remain competitive in architectures such as VGG-16. Relevance-based scores ask what a channel says about the task; pruning asks whether the network still needs that channel when its peers remain available.

模型剪枝通道重要性双轴分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。