压缩RWKV模型，让大语言模型在手机等设备上运行
RWKV-edge: Deeply Compressed RWKV for Resource-Constrained Devices
- 针对RWKV架构设计压缩技术，从结构优化到训练后压缩
- 内存减少3.4至5倍，精度损失极小
- 适合移动端、机器人等资源受限场景
为在移动机器人和智能手机等资源受限平台部署大语言模型，非变换器类模型已取得显著进展。近期基于循环神经网络的RWKV模型展现出优异的计算效率，但其参数量仍较高，限制了实际部署。本文提出一套针对RWKV架构的压缩技术，涵盖模型结构优化与训练后压缩。结合各项技术，可使RWKV模型内存占用减少3.4至5倍，精度损失微乎其微；相较于同等精度的Transformer模型，本方案内存需求降低4倍。
原文摘要 · Abstract (English)
To deploy LLMs on resource-contained platforms such as mobile robots and smartphones, non-transformers LLMs have achieved major breakthroughs. Recently, a novel RNN-based LLM family, Repentance Weighted Key Value (RWKV) has shown strong computational efficiency; nevertheless, RWKV models still have high parameter counts which limited their deployment. In this paper, we propose a suite of compression techniques, ranging from model architecture optimizations to post-training compression, tailored to the RWKV architecture. Combined, our techniques reduce the memory footprint of RWKV models by 3.4x -- 5x with only negligible degradation in accuracy; compared to transformer LLMs with similar accuracy, our models require 4x less memory footprint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。