通过信息引导剪枝与渐进切片，实现高精度低计算量模型压缩。
MI-to-Mid Distilled Compression (M2M-DC): An Hybrid-Information-Guided-Block Pruning with Progressive Inner Slicing Approach to Model Compression
- 基于标签感知互信息排序残差块，结构化剪枝低信息单元。
- ResNet-18仅需309万参数达85.46%准确率，计算量降低至教师的63%。
- 适配残差与倒置残差结构，适合部署场景的轻量化模型设计。
我们提出MI-to-Mid Distilled Compression（M2M-DC），一种双尺度、形状安全的模型压缩框架，将信息引导的块剪枝与渐进内切片及分阶段知识蒸馏相结合。首先，基于标签感知互信息（MI）信号对残差（或倒置残差）块进行排序，并移除信息最少的单元（训练后结构化剪枝）。随后交替执行短周期知识蒸馏与阶段一致的残差安全通道切片：(i) 阶段“平面”（共切片 conv2 输出通道与下采样路径及下一阶段输入），(ii) 可选的中通道裁剪（conv1 输出 / bn1 / conv2 输入）。该方法针对互补冗余、完整计算模块及阶段内宽度，同时保留残差形状不变性。在CIFAR-100上，M2M-DC实现了清晰的精度-算力前沿。对于ResNet-18，达到85.46% Top-1准确率，仅需3.09M参数和0.0139 GMacs（相比教师模型减少72%参数、63% GMacs；三次种子平均最终准确率为85.29%）。ResNet-34达到85.02% Top-1准确率，参数5.46M，GMacs 0.0195（74%/74% vs. 教师；平均84.62%）。扩展至倒置残差结构，MobileNetV2在1.71M参数（27%）和0.0186 conv GMacs（24%）下取得平均68.54% Top-1准确率，较教师模型66.03%提升+2.5点。由于M2M-DC仅暴露薄而架构感知的接口（块、阶段、下采样/跳跃连接），其可泛化至多种残差CNN，并延伸至倒置残差族，仅需少量合法化规则。最终形成一套紧凑、实用的部署就绪模型压缩方案，在极低计算开销下匹配或超越教师模型精度。
原文摘要 · Abstract (English)
We introduce MI-to-Mid Distilled Compression (M2M-DC), a two-scale, shape-safe compression framework that interleaves information-guided block pruning with progressive inner slicing and staged knowledge distillation (KD). First, M2M-DC ranks residual (or inverted-residual) blocks by a label-aware mutual information (MI) signal and removes the least informative units (structured prune-after-training). It then alternates short KD phases with stage-coherent, residual-safe channel slicing: (i) stage "planes" (co-slicing conv2 out-channels with the downsample path and next-stage inputs), and (ii) an optional mid-channel trim (conv1 out / bn1 / conv2 in). This targets complementary redundancy, whole computational motifs and within-stage width while preserving residual shape invariants. On CIFAR-100, M2M-DC yields a clean accuracy-compute frontier. For ResNet-18, we obtain 85.46% Top-1 with 3.09M parameters and 0.0139 GMacs (72% params, 63% GMacs vs. teacher; mean final 85.29% over three seeds). For ResNet-34, we reach 85.02% Top-1 with 5.46M params and 0.0195 GMacs (74% / 74% vs. teacher; mean final 84.62%). Extending to inverted-residuals, MobileNetV2 achieves a mean final 68.54% Top-1 at 1.71M params (27%) and 0.0186 conv GMacs (24%), improving over the teacher's 66.03% by +2.5 points across three seeds. Because M2M-DC exposes only a thin, architecture-aware interface (blocks, stages, and down sample/skip wiring), it generalizes across residual CNNs and extends to inverted-residual families with minor legalization rules. The result is a compact, practical recipe for deployment-ready models that match or surpass teacher accuracy at a fraction of the compute.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。