根据词元重要性动态分配算力，显著降低Transformer模型的功耗和延迟。
Adaptive Model Compression (AMC): Saliency-Driven Resource Allocation for Ultra-Low-Power Transformer Inference
- 按词元重要性分层处理，关键信息用高精度，次要信息降秩减位宽。
- 在45nm芯片上能降低59.2%能耗，吞吐量提升2.24倍。
- 适合对能效敏感的移动端部署，仅牺牲3.6%准确率。
在资源受限的边缘设备上部署大规模Transformer模型仍具挑战，因其静态推理对简单与复杂词元均采用统一计算强度，导致高能耗与内存开销。为此，我们提出自适应模型压缩（AMC），一种基于词元显著性的框架，可动态分配硬件资源。通过多层级架构，系统识别出关键高显著性信息并进行全精度处理，同时对低显著性数据激进地降低秩和位宽。实验表明，在45nm CMOS硬件上，AMC实现系统能耗降低59.2%，吞吐量提升2.24倍，仅以3.6%的准确率损失维持鲁棒性能，有效延长移动设备电池寿命。
原文摘要 · Abstract (English)
Deploying large-scale transformer models on resource-constrained edge devices remains a challenge due to the high energy and memory overhead inherent in static inference, which processes simple and complex tokens with uniform intensity. To address this, we propose Adaptive Model Compression (AMC), a saliency-driven framework that dynamically allocates hardware resources based on token importance. By implementing a multi-tier architecture, our system identifies critical high-saliency information for full-precision processing while aggressively reducing the rank and bit-width of less significant data. Experimental results demonstrate that AMC achieves a 59.2% reduction in system energy and a 2.24x increase in throughput on 45nm CMOS hardware. This approach effectively extends the battery life of mobile devices by utilizing high-definition compute only where necessary, maintaining robust performance with a marginal 3.6% accuracy trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。