通过分段管理内存，让边缘设备跑大模型更省电。
SHIELD: A Segmented Hierarchical Memory Architecture for Energy-Efficient LLM Inference on Edge NPUs
- 按激活数据生命周期分段管理，区分临时与持久数据
- 关闭临时数据刷新，对持久数据放宽刷新频率
- 在多个模型上省电35%，且不影响评测准确率
大语言模型在边缘神经处理器上的推理受限于片上内存容量。尽管高密度嵌入式DRAM（eDRAM）适合存储激活数据，但其周期性刷新会消耗大量能量。以往工作主要关注减少片外通信或优化持久键值（KV）缓存的刷新策略，而对临时且容错的查询与注意力输出（QO）激活则较少关注。我们提出SHIELD，一种生命周期感知的分段eDRAM架构，同时利用bfloat16（BF16）激活数据的时间驻留特性与位级敏感性。该架构将符号位和指数位与尾数分离，禁用临时QO尾数的刷新，并对持久KV尾数采用宽松刷新机制。在多个大语言模型及推理场景下，相比标准刷新基线，SHIELD将eDRAM刷新能耗降低35%，同时在WikiText-2、PIQA和ARC-Easy数据集上保持精度不变。
原文摘要 · Abstract (English)
Large Language Model (LLM) inference on edge Neural Processing Units (NPUs) is fundamentally constrained by limited on-chip memory capacity. Although high-density embedded DRAM (eDRAM) is attractive for storing activation workspaces, its periodic refresh consumes substantial energy. Prior work has primarily focused on reducing off-chip traffic or optimizing refresh for persistent Key-Value (KV) caches, while transient and error-resilient Query and Attention Output (QO) activations are largely overlooked. We propose SHIELD, a lifecycle-aware segmented eDRAM architecture that jointly exploits temporal residency and bit-level sensitivity in bfloat16 (BF16) activations. SHIELD isolates the sign and exponent fields from the mantissa, disables refresh for transient QO mantissas, and applies relaxed refresh to persistent KV mantissas. Across multiple LLMs and inference scenarios, SHIELD reduces eDRAM refresh energy by 35% relative to a standard-refresh baseline while preserving accuracy on WikiText-2, PIQA, and ARC-Easy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。