提出新型注意力机制PLGA，可精确包含原注意力，且在推理时出现输出坍缩现象。
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
- 用可学习的幂律张量构造输入相关注意力算子，替代固定形式的点积注意力。
- 推理阶段输出完全不变，等价于使用常数算子的简化注意力，实测波动低于10^-6。
- 适合关注注意力机制理论严谨性与推理稳定性研究的模型开发者和理论研究者。
基于幂律解码器表征的大语言模型（PLDR-LLM）及其注意力机制——幂律图注意力（PLGA），将传统缩放点积注意力（SDPA）固定的双线性形式替换为由正张量 $A_{LM}$ 通过逐元素幂律构建的可学习、输入生成的双线性算子 $G_{LM}$。该架构完整定义并经验证；论断分为定理、条件定理、测量结果或猜想。无条件成立：当 $G_{LM}=I$ 时，PLGA 精确包含 SDPA；$A_{LM}$ 和 $A_P$ 严格正，$A_{LM}$ 具有 Perron-Frobenius 结构；有向无环图正则项具有 NOTEARS 的路径计数形式，但正性阻碍精确无环；在非共振条件下（标准旋转频率满足），可通过交换子准则识别保持相对位置依赖的算子。推论坍缩定理表明：演绎输出的精确输入不变性导致推理退化为使用常数算子的广义 SDPA。实测显示相对波动低于 $10^{-6}$；扰动界可量化但无法保证缓存推理的正确性；组装代理未能覆盖解码边界。一个三阶段机制（旋转旋转变换、集中、行映射压缩）在公开检查点上被测量验证。采用全局格拉姆矩阵的块训练与评分策略明确指定目标暴露；在测试样本上，块内与顺序评分选择相同答案，且在公布的 TruthfulQA 概率质量指标上每项差异不超过 $5\times10^{-5}$。自组织临界性作为现象学框架引入，内在序参量使开放问题可证伪；部分核心证明已在 Lean 4 中机器验证。
原文摘要 · Abstract (English)
The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA), replace the fixed bilinear form of scaled dot-product attention (SDPA) with a learned, input-generated bilinear operator $G_{LM}$, built from a positive tensor $A_{LM}$ by elementwise power laws. The architecture is fully specified, verified against pinned reference releases; claims are labeled theorem, conditional theorem, measurement, or conjecture. Unconditionally: PLGA contains SDPA exactly at $G_{LM}=I$; $A_{LM}$ and $A_P$ are strictly entrywise positive, with Perron-Frobenius structure on $A_{LM}$; the DAG regularizer has the NOTEARS walk-counting form and positivity obstructs exact acyclicity; and, under nonresonance (satisfied by standard rotary frequencies), a commutant criterion identifies which operators preserve relative-position dependence. An inference-collapse theorem: exact input invariance of deductive outputs collapses inference to generalized SDPA with a constant operator. Measured invariance: relative fluctuations of $10^{-6}$ and below; perturbation bounds quantify but do not certify cached inference; the assembled proxy misses the decoding margin. A conditional three-stage mechanism (rotary twirl, concentration, row-map contraction) is measured on a released checkpoint. Blockwise training and scoring under the global Gram are stated with explicit target exposure; on tested samples, block and sequential scoring select identical answers and agree on the published TruthfulQA probability-mass metric within $5\times 10^{-5}$ per item. Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures. Selected proof cores are machine-checked in Lean 4.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。