arXiv:2411.17367cs.ARcs.LG2024-11被引 1

让Transformer在模拟存内计算硬件上高效适配,仅需轻量模块无需重训练。

Efficient transformer adaptation for analog in-memory computing via low-rank adapters

  • 用低秩适配模块替代重训,固定模拟权重实现软硬协同
  • 在SQuAD和GLUE上验证,大模型与指令微调均有效
  • 混合架构每层仅增4%开销,适合边缘部署

模拟存内计算(AIMC)为缓解冯诺依曼瓶颈提供新路径。但将Transformer部署于AIMC面临挑战:其需跨任务灵活适应,而静态向量-矩阵乘法权重必须以权值驻留方式映射至模拟器件。传统硬件感知训练需重训全模型,且重编程模拟器件耗时耗能。为此,我们提出模拟硬件感知低秩适配(AHWA-LoRA)训练方法,保持模拟权重为元权重,引入轻量外部LoRA模块实现硬件与任务双重适配。在SQuAD v1.1与GLUE基准测试中验证,支持更大模型、指令微调及强化学习。进一步评估了结合RISC-V可编程多核加速器的流水线策略,在平衡AIMC单元延迟与数字LoRA处理的前提下,该混合架构实现高效推理,相比纯AIMC方案每层仅增加4%开销。

原文摘要 · Abstract (English)

Analog In-Memory Computing (AIMC) offers a promising solution to the von Neumann bottleneck. However, deploying transformer models on AIMC remains challenging due to their inherent need for flexibility and adaptability across diverse tasks. For the benefits of AIMC to be fully realized, weights of static vector-matrix multiplications must be mapped and programmed to analog devices in a weight-stationary manner. This poses two challenges for adapting a base network to hardware and downstream tasks: (i) conventional analog hardware-aware (AHWA) training requires retraining the entire model, and (ii) reprogramming analog devices is both time- and energy-intensive. To address these issues, we propose Analog Hardware-Aware Low-Rank Adaptation (AHWA-LoRA) training, a novel approach for efficiently adapting transformers to AIMC hardware. AHWA-LoRA training keeps the analog weights fixed as meta-weights and introduces lightweight external LoRA modules for both hardware and task adaptation. We validate AHWA-LoRA training on SQuAD v1.1 and the GLUE benchmark, demonstrate its scalability to larger models, and show its effectiveness in instruction tuning and reinforcement learning. We further evaluate a practical deployment scenario that balances AIMC tile latency with digital LoRA processing using optimized pipeline strategies, with RISC-V-based programmable multi-core accelerators. This hybrid architecture achieves efficient transformer inference with only a 4% per-layer overhead compared to a fully AIMC implementation.

模拟计算TransformerLoRA硬件协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。