通过分块选神经元，显著降低视觉语言模型在边缘设备的I/O开销。
VLM in a flash: I/O-Efficient Sparsification of Vision-Language Model via Neuron Chunking
- 按内存连续块分组选择重要神经元，结合访问成本优化
- 在Jetson Orin Nano上提升I/O效率4.65倍，AGX Orin上达5.76倍
- 适合资源受限的边缘部署场景，尤其关注存储性能的模型
边缘部署大型视觉语言模型（VLMs）越来越依赖基于闪存的权重卸载，激活稀疏化用于降低I/O开销。然而，传统稀疏化方法仍以模型为中心，仅根据激活幅度选择神经元，忽视了访问模式对闪存性能的影响。我们提出Neuron Chunking，一种面向I/O高效的稀疏化策略，该策略基于块（即内存中连续的一组神经元）操作，并将神经元重要性与存储访问成本相结合。该方法通过轻量级抽象建模访问连续性带来的I/O延迟，选择具有高效用的块（即神经元重要性除以估计延迟）。通过将稀疏化决策与底层存储行为对齐，Neuron Chunking在Jetson Orin Nano和Jetson AGX Orin上分别实现高达4.65倍和5.76倍的I/O效率提升。
原文摘要 · Abstract (English)
Edge deployment of large Vision-Language Models (VLMs) increasingly relies on flash-based weight offloading, where activation sparsification is used to reduce I/O overhead. However, conventional sparsification remains model-centric, selecting neurons solely by activation magnitude and neglecting how access patterns influence flash performance. We present Neuron Chunking, an I/O-efficient sparsification strategy that operates on chunks (i.e., groups of contiguous neurons in memory) and couples neuron importance with storage access cost. The method models I/O latency through a lightweight abstraction of access contiguity and selects chunks with high utility, defined as neuron importance normalized by estimated latency. By aligning sparsification decisions with the underlying storage behavior, Neuron Chunking improves I/O efficiency by up to 4.65x and 5.76x on Jetson Orin Nano and Jetson AGX Orin, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。