arXiv:2502.13502cs.CLcs.AI2025-02被引 1

PLDR-LLM学会用张量替代神经网络,推理更快且结果不变。

PLDR-LLMs Learn A Generalizable Tensor Operator That Can Replace Its Own Deep Neural Net At Inference

  • 通过幂律解码器学习不变张量,取代原神经网络推理
  • 缓存后输出误差小于10^-15,零样本性能无损
  • 适合追求高效推理的模型部署与理论研究者

我们证明,基于幂律解码器表示的大型语言模型(PLDR-LLM)是一种基础模型,其推导输出在小扰动下保持张量不变。该模型学习到推导输出的奇异性条件,使得先前推导出的能量-曲率张量 $oldsymbol{G}_{LM}$ 可在推理阶段替代生成推导输出的幂律图注意力(PLGA)深度神经网络。我们演示了可直接实现 $oldsymbol{G}_{LM}$ 缓存(G-cache)与键值缓存(KV-cache),显著提升推理速度。推导输出具有极高保真度,缓存后输出的均方根误差(RMSE)和行列式值在15位小数内一致,零样本基准分数不变。消融实验表明,学习到的推导输出在损失和准确率上与使用转移、随机初始化或恒等张量作为常数算子的模型有显著差异;采用缩放点积注意力(SDPA)的LLM是PLDR-LLM的特例,其中 $oldsymbol{G}_{LM}$ 被预设为恒等矩阵。观察到的不变性特征引入了训练与推理阶段的新不对称性,并总结了学习到的奇异性条件下推导输出的共性特征。我们提供了包含KV-cache与G-cache的PLDR-LLM训练与推理框架实现。

原文摘要 · Abstract (English)

We show that Large Language Model from Power Law Decoder Representations (PLDR-LLM) is a foundational model whose deductive outputs are invariant tensors up to a small perturbation. PLDR-LLM learns a singularity condition for the deductive outputs that enable the once-inferred energy-curvature tensor $\mathbf{G}_{LM}$ to replace the deep neural network of power law graph attention (PLGA) generating the deductive outputs at inference. We demonstrate that a cache for $\mathbf{G}_{LM}$ (G-cache) and KV-cache can be implemented in a straightforward manner to improve the inference time. The invariance and generalizable nature of deductive outputs is at a very high fidelity where deductive outputs have same RMSE and determinant values up to 15 decimal places after caching, and zero-shot benchmark scores remain unchanged. Ablation studies show that learned deductive outputs have distinct loss and accuracy characteristics from models pretrained with transferred, randomly initialized or identity tensors as a constant tensor operator and an LLM with scaled-dot product attention (SDPA) is a special case of PLDR-LLM where $\mathbf{G}_{LM}$ is predefined as identity. The observed invariance characteristic introduces a novel asymmetry between training and inference phases with caching. We outline observed common characteristics of the deductive outputs for the learned singularity condition. We provide an implementation of a training and inference framework for PLDR-LLM with KV-cache and G-cache.

大模型推理张量替代缓存优化幂律结构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。