用可自适应的分块结构矩阵，让大模型推理更快更省资源。
BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference
- 提出分块自适应结构矩阵，灵活学习权重中的高效模式。
- 中型模型压缩70%~40%计算量，大型模型压缩2倍且损失最小。
- 适合追求高效推理的大模型部署，尤其在资源受限场景。
大规模基础模型在语言和视觉任务中表现出色，但其深层网络中的密集矩阵-向量运算带来了巨大的推理计算负担。为解决这一问题，我们提出块级自适应结构（BLAST）矩阵，旨在学习并利用深度学习模型线性层权重矩阵中存在的高效结构。相比现有结构化矩阵，BLAST具有更强灵活性，能表示从数据中学习或由预存权重计算出的多种结构。实验表明，该方法在语言与视觉任务中均具高效性：(i) 对ViT和GPT-2等中型模型，使用BLAST权重训练可分别降低70%和40%的复杂度，同时提升性能；(ii) 对Llama-7B和DiT-XL等大型基础模型，实现2倍压缩，且性能下降低于所有对比的结构化矩阵。代码已开源：https://github.com/changwoolee/BLAST。
原文摘要 · Abstract (English)
Large-scale foundation models have demonstrated exceptional performance in language and vision tasks. However, the numerous dense matrix-vector operations involved in these large networks pose significant computational challenges during inference. To address these challenges, we introduce the Block-Level Adaptive STructured (BLAST) matrix, designed to learn and leverage efficient structures prevalent in the weight matrices of linear layers within deep learning models. Compared to existing structured matrices, the BLAST matrix offers substantial flexibility, as it can represent various types of structures that are either learned from data or computed from pre-existing weight matrices. We demonstrate the efficiency of using the BLAST matrix for compressing both language and vision tasks, showing that (i) for medium-sized models such as ViT and GPT-2, training with BLAST weights boosts performance while reducing complexity by 70% and 40%, respectively; and (ii) for large foundation models such as Llama-7B and DiT-XL, the BLAST matrix achieves a 2x compression while exhibiting the lowest performance degradation among all tested structured matrices. Our code is available at https://github.com/changwoolee/BLAST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。