arXiv:2601.17987cs.LGcs.CV2026-01

系统研究模型精简三要素:收敛、剪枝与量化,找最简稳定架构。

Systematic Characterization of Minimal Deep Learning Architectures: A Unified Analysis of Convergence, Pruning, and Quantization

  • 构建统一分析框架,跨网络结构测试学习行为
  • 发现参数量减少60%仍可稳定训练,复杂任务更依赖高精度
  • 为低资源部署提供可量化选型依据,适合模型压缩研究者

深度学习模型在分类任务中表现优异,但找到能可靠完成任务的最小架构仍具挑战。本文提出一种计算方法,系统探索收敛性、剪枝敏感性和量化鲁棒性之间的关系。通过大规模架构设计扫描,评估代表性模型在图像分类任务(逐步增加复杂度)中的表现,涵盖深度神经网络、卷积神经网络和视觉变压器。结果表明,尽管架构多样,性能基本不变,学习动态一致呈现三种状态:不稳定、学习、过拟合。进一步揭示稳定学习所需的最少可训练参数,识别出不同的收敛与剪枝阶段,并量化数值精度降低对可训练参数的影响。符合直觉的是,更深的架构对剪枝更具鲁棒性(参数冗余高达60%),而较少参数的模型受量化影响更大,且在更难的数据集上影响更显著。这些发现为在剪枝和低精度约束下选择紧凑、稳定的图像分类模型提供了可操作的指导。

原文摘要 · Abstract (English)

Deep learning networks excel at classification, yet identifying minimal architectures that reliably solve a task remains challenging. We present a computational methodology for systematically exploring and analyzing the relationships among convergence, pruning, and quantization. The workflow first performs a structured design sweep across a large set of architectures, then evaluates convergence behavior, pruning sensitivity, and quantization robustness on representative models. Focusing on well-known image classification of increasing complexity, and across Deep Neural Networks, Convolutional Neural Networks, and Vision Transformers, our initial results show that, despite architectural diversity, performance is largely invariant and learning dynamics consistently exhibit three regimes: unstable, learning, and overfitting. We further characterize the minimal learnable parameters required for stable learning, uncover distinct convergence and pruning phases, and quantify the effect of reduced numeric precision on trainable parameters. Aligning with intuition, the results confirm that deeper architectures are more resilient to pruning than shallower ones, with parameter redundancy as high as 60%, and quantization impacts models with fewer learnable parameters more severely and has a larger effect on harder image datasets. These findings provide actionable guidance for selecting compact, stable models under pruning and low-precision constraints in image classification.

模型压缩量化剪枝架构分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。