通过选择性加载与计算神经元，实现边缘设备上大模型高效推理
SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs
- 基于离线分析筛选关键神经元，仅加载和计算必要部分
- 内存占用减少显著，推理性能基本不受影响
- 适合资源受限的移动端或嵌入式大模型部署
大型语言模型（LLMs）在自然语言处理任务中表现出色，但其高计算与内存需求给资源受限的边缘设备部署带来挑战。现有模型压缩与优化方法多依赖粗粒度剪枝或量化，常导致精度下降或需重新训练。本文提出SelectInfer，一种基于神经元级别的优化框架，通过选择性加载与计算实现边缘设备上的高效推理。该方法利用离线的LLM分析器识别任务相关与通用神经元，实施两项优化：选择性加载（仅加载关键神经元以降低内存占用），以及运行时动态计算最相关神经元。多数据集评估表明，SelectInfer在保持任务性能的同时，显著降低内存占用与计算量，是推动大模型在边缘设备落地的重要实践。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands pose significant challenges for deployment on resource-constrained edge devices. Existing approaches to model compression and optimization often rely on coarse-grained pruning or quantization, which can compromise accuracy or require re-training and fine-tuning. In this work, we introduce SelectInfer, a neuron-level optimization framework that enables efficient LLM inference on edge devices through selective neuron loading and computation. By profiling and identifying both task-specific and general-purpose neurons using an offline LLM profiler, SelectInfer implements two key optimizations: selective loading, which reduces memory footprint by selectively loading a subset of neurons that were identified to be most important during the offline stage, and selective computation, which dynamically computes only the most relevant neurons at runtime. Evaluation across multiple datasets shows that SelectInfer achieves significant reductions in memory footprint and computation while preserving task performance, making it a practical step towards enabling LLM deployment on edge devices
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。