提出BALF框架，实现无需微调的高效模型压缩。
BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression
- 基于白化理论扩展激活感知低秩分解至卷积层
- 在相同计算量下比SVD基线提升准确率
- 适合追求轻量化部署的视觉模型开发者
激活感知的低秩分解技术虽能实现良好压缩效果，但通常仅限于线性层，而现有基于白化的理论常隐含对激活的满秩假设。本文提出一种层表示框架，将激活感知因子分解扩展至标准与分组卷积层。在此框架下，我们的白化公式更通用，自然涵盖秩亏激活，并生成最优低秩投影，达到对层激活的最佳低秩逼近重建误差。由此得到的奇异谱提供每层失真闭式代理，结合拉格朗日松弛法，在显式计算量或参数量预算下实现逐层秩分配，开销极小。上述组件构成端到端的BALF压缩流水线。在CIFAR-10和ImageNet-1K上的实验表明，相比基于SVD的基线方法，BALF在相同FLOP或参数量目标下普遍获得更高精度，且与其它免微调压缩方法保持竞争力。
原文摘要 · Abstract (English)
Activation-aware low-rank factorization techniques yield strong compression results but are generally confined to linear layers, while existing whitening-based theory typically makes an implicit full-rank assumption on activations. We introduce a layer representation framework that extends activation-aware factorization beyond linear layers, including standard and grouped convolutions. Within this framework, our whitening-based formulation is more general than prior ones, naturally covering rank-deficient activations, and yields an optimal low-rank projection that attains the reconstruction error of the best low-rank approximation to layer activations. The resulting singular spectrum provides a closed-form per-layer distortion proxy, which we use to allocate per-layer ranks under explicit FLOP or parameter-count budgets via a Lagrangian relaxation with negligible overhead. Together, these components form BALF, an end-to-end pipeline for efficient vision model compression. Across CNNs and vision transformers on CIFAR-10 and ImageNet-1K, BALF generally achieves higher accuracy than SVD-based factorization baselines at matched FLOP or parameter count targets and remains competitive with other fine-tuning-free compression techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。