arXiv:2410.19274cs.LGcs.AI2024-10被引 7

通过神经元共激活链接,提升手机端大模型推理速度。

Neuralink: Fast LLM Inference on Smartphones with Neuron Co-Activation Linking

  • 根据神经元共激活模式优化闪存中的存储布局。
  • 在多种手机上平均降低1.49倍端到端延迟。
  • 适合移动端大模型部署,尤其关注低延迟场景。

大语言模型(LLMs)在多个领域取得显著成果,但在移动设备上部署仍面临巨大挑战,主要源于其庞大的计算与内存需求。尽管已有轻量级LLM适配移动环境,但会牺牲模型精度。基于稀疏性的方法通过仅将相关神经元加载到DRAM来减少内存占用,同时保留完整模型于外部存储(如闪存)。然而,这类方法受限于频繁的I/O操作,尤其在智能手机严苛的每秒输入输出次数(IOPS)约束下表现不佳。本文提出Neuralink,一种通过优化闪存中神经元布局加速手机端LLM推理的新方法。Neuralink利用神经元共激活概念,将频繁共同激活的神经元关联起来,以支持连续读取并提升I/O效率。该方案包含离线阶段:基于共激活模式重新组织神经元位置;以及在线阶段:采用定制化数据访问与缓存策略,适配硬件特性。在多种智能手机和LLM上的评估显示,Neuralink相较当前最优方案平均实现1.49倍的端到端延迟降低。作为首个在稀疏性框架下优化存储布局的方案,Neuralink探索了稀疏驱动算法与存储系统协同设计的新优化空间。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have achieved remarkable success across various domains, yet deploying them on mobile devices remains an arduous challenge due to their extensive computational and memory demands. While lightweight LLMs have been developed to fit mobile environments, they suffer from degraded model accuracy. In contrast, sparsity-based techniques minimize DRAM usage by selectively transferring only relevant neurons to DRAM while retaining the full model in external storage, such as flash. However, such approaches are critically limited by numerous I/O operations, particularly on smartphones with severe IOPS constraints. In this paper, we propose Neuralink, a novel approach that accelerates LLM inference on smartphones by optimizing neuron placement in flash memory. Neuralink leverages the concept of Neuron Co-Activation, where neurons frequently activated together are linked to facilitate continuous read access and optimize I/O efficiency. Our approach incorporates a two-stage solution: an offline stage that reorganizes neuron placement based on co-activation patterns, and an online stage that employs tailored data access and caching strategies to align well with hardware characteristics. Evaluations conducted on a variety of smartphones and LLMs demonstrate that Neuralink achieves on average $1.49\times$ improvements in end-to-end latency compared to the state-of-the-art. As the first solution to optimize storage placement under sparsity, Neuralink explores a new optimization space at the intersection of sparsity-driven algorithm and storage-level system co-design for LLM inference.

大模型推理移动端优化神经元共激活闪存调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。