用生成模型思想优化残差网络,参数减一半还更准
MFI-ResNet: Efficient ResNet Architecture Optimization via MeanFlow Compression and Selective Incubation
- 将残差块压缩为单个流场模块,再选择性恢复前几阶段结构
- 在CIFAR-10/100上参数减少超45%,准确率反而提升0.17%~0.23%
- 为判别学习与生成建模的关联提供新视角,适合模型压缩研究者
ResNet凭借残差连接机制在计算机视觉中取得巨大成功,可被视为常微分方程(ODE)的离散形式。每个残差阶段内的多层结构本质上是对该阶段特征变换的多步离散迭代。近期提出的流匹配模型MeanFlow可通过学习均值速度场实现单步生成建模。受此启发,本文提出均值流孵化残差网络(MFI-ResNet),采用压缩-扩展策略同时提升参数效率与判别性能。在压缩阶段,将每阶段多层结构简化为一个或两个MeanFlow模块,构建轻量级元模型;在扩展阶段,对前三个阶段应用选择性孵化策略,还原为基准ResNet的残差块配置,而保留最后一阶段为MeanFlow形式,并进行微调。实验表明,在CIFAR-10和CIFAR-100数据集上,相比ResNet-50,MFI-ResNet分别减少46.28%和45.59%参数,同时准确率提升0.23%和0.17%。结果表明,生成式流场能有效刻画ResNet中的特征变换过程,为生成建模与判别学习的关系提供了新理解。
原文摘要 · Abstract (English)
ResNet has achieved tremendous success in computer vision through its residual connection mechanism. ResNet can be viewed as a discretized form of ordinary differential equations (ODEs). From this perspective, the multiple residual blocks within a single ResNet stage essentially perform multi-step discrete iterations of the feature transformation for that stage. The recently proposed flow matching model, MeanFlow, enables one-step generative modeling by learning the mean velocity field to transform distributions. Inspired by this, we propose MeanFlow-Incubated ResNet (MFI-ResNet), which employs a compression-expansion strategy to jointly improve parameter efficiency and discriminative performance. In the compression phase, we simplify the multi-layer structure within each ResNet stage to one or two MeanFlow modules to construct a lightweight meta model. In the expansion phase, we apply a selective incubation strategy to the first three stages, expanding them to match the residual block configuration of the baseline ResNet model, while keeping the last stage in MeanFlow form, and fine-tune the incubated model. Experimental results show that on CIFAR-10 and CIFAR-100 datasets, MFI-ResNet achieves remarkable parameter efficiency, reducing parameters by 46.28% and 45.59% compared to ResNet-50, while still improving accuracy by 0.23% and 0.17%, respectively. This demonstrates that generative flow-fields can effectively characterize the feature transformation process in ResNet, providing a new perspective for understanding the relationship between generative modeling and discriminative learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。