通过动态分配计算资源，让Transformer模型推理更快更准。
Adaptive Rank Allocation: Speeding Up Modern Transformers with RaNA Adapters
- 用低秩分解+自适应掩码，在不依赖稀疏激活的前提下分配算力。
- 在减少约44%计算量时，困惑度降低7点，准确率提升8个百分点。
- 适配MLP和注意力模块，无需昂贵的掩码机制，通用性强。
大型语言模型在推理阶段计算开销巨大。现有神经元自适应方法虽能提速，但在现代Transformer中存在局限：依赖稀疏激活、难以适配注意力层、需高成本掩码技术。为此，我们提出自适应秩分配框架及RaNA适配器。RaNA利用低秩适配器，通过对线性层应用低秩分解与自适应掩码，高效分配计算资源，不依赖激活稀疏性。该方法可通用部署于MLP及注意力模块的线性部分，无需昂贵掩码器。相比神经元适配器，在主流Transformer架构中降低约44%浮点运算量时，困惑度最高下降7点，准确率最高提升8个百分点。结果表明，RaNA是提升现代Transformer推理效率的稳健方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are computationally intensive, particularly during inference. Neuron-adaptive techniques, which selectively activate neurons in Multi-Layer Perceptron (MLP) layers, offer some speedups but suffer from limitations in modern Transformers. These include reliance on sparse activations, incompatibility with attention layers, and the use of costly neuron masking techniques. To address these issues, we propose the Adaptive Rank Allocation framework and introduce the Rank and Neuron Allocator (RaNA) adapter. RaNA adapters leverage rank adapters, which operate on linear layers by applying both low-rank matrix decompositions and adaptive masking to efficiently allocate compute without depending on activation sparsity. This enables RaNA to be generally applied to MLPs and linear components of attention modules, while eliminating the need for expensive maskers found in neuron-adaptive methods. Notably, when compared to neuron adapters, RaNA improves perplexity by up to 7 points and increases accuracy by up to 8 percentage-points when reducing FLOPs by $\sim$44% in state-of-the-art Transformer architectures. These results position RaNA as a robust solution for improving inference efficiency in modern Transformer architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。