通过前瞻式精度调度，大幅降低大模型推理误差。
LAMP: Look-Ahead Mixed-Precision Inference of Large Language Models
- 分析组合函数误差传播，选择性提升部分计算精度。
- 低重计算率下，准确率最高提升两个数量级。
- 适合追求高精度与低资源消耗的本地化大模型部署。
混合精度计算是当前人工智能的重要特征，推动大语言模型向高效、本地部署方向发展。本文聚焦于复合函数 $f(g(oldsymbol{x}))$ 的浮点计算问题，基于对组合函数中舍入误差的分析，提出一种自适应策略：仅选择 $g(oldsymbol{x})$ 的一小部分组件进行更高精度计算，其余部分可保持低精度。该策略可扩展至Transformer内部不同组合结构，并在GPT-2模型上进行了数值验证。结果表明，即使重计算率极低，也能实现高达两个数量级的精度提升。
原文摘要 · Abstract (English)
Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article addresses the floating-point computation of compositionally-rich functions, concentrating on transformer inference. Based on the rounding error analysis of a composition $f(g(\mathrm{x}))$, we provide an adaptive strategy that selects a small subset of components of $g(\mathrm{x})$ to be computed more accurately while all other computations can be carried out with lower accuracy. We then explain how this strategy can be applied to different compositions within a transformer and illustrate its overall effect on transformer inference. We study the effectiveness of this algorithm numerically on GPT-2 models and demonstrate that already very low recomputation rates allow for improvements of up to two orders of magnitude in accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。