通过注意力感知联合张量压缩，让大模型更小更快还更准。
LatentLLM: Attention-Aware Joint Tensor Compression
- 基于注意力机制改进张量分解，实现全局感知的压缩。
- 在降低隐空间维度时，准确率优于现有压缩方法。
- 适合需要高效推理的大模型部署场景。
当前的大规模基础模型（如大语言模型 LLMs 与多模态模型 LMMs）需要大量计算和内存资源。本文提出一种新框架，将这些模型转化为低维隐空间结构。该方法将局部激活感知的张量分解扩展为全局注意力感知的联合张量分解。在降低隐空间维度以实现计算与内存效率的同时，显著提升模型准确率。该方法在多个基准测试中表现出色，包括多模态推理任务。
原文摘要 · Abstract (English)
Modern foundation models such as large language models (LLMs) and large multi-modal models (LMMs) require a massive amount of computational and memory resources. We propose a new framework to convert such LLMs/LMMs into a reduced-dimension latent structure. Our method extends a local activation-aware tensor decomposition to a global attention-aware joint tensor de-composition. Our framework can significantly improve the model accuracy over the existing model compression methods when reducing the latent dimension to realize computationally/memory-efficient LLMs/LLMs. We show the benefit on several benchmark including multi-modal reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。