用数学方法推导出四种高效推理优化，显著降低显存占用与计算开销。
MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel
- 基于数组数学推导出单查询解码的最优内存访问模式。
- 实现零误差精确计算，显存流量减少至约4倍批量大小的理论下界。
- 适用于追求极致推理效率的模型部署场景，尤其适合GQA/MQA架构。
本文基于数组数学（MoA），在固定解码步骤查询行索引的前提下，从前向传播的语义范式（DNF）出发，推导出四种内存最优的Transformer注意力推理优化方案：(1) 单查询解码的DNF，通过ψ-约化代数消去K⊤缓存，实测误差‖err‖∞≤2×10⁻⁷，动态内存访问量为 (d_k + n d_k + n d_v + d_v) × 4B；(2) 采用操作范式（ONF）步长算术与硬件合并内存访问的C/OpenACC GPU内核，验证误差‖err‖∞=0（精确IEEE-754浮点运算）；(3) 基于MoA连接符#的多步KV缓存机制，每步追加复杂度为O(d_k + d_v)；(4) 通过ψ-选择推导出分组查询注意力（GQA）与多查询注意力（MQA），证明可实现KV通信量缩小至h_q/h_kv倍。所有程序均通过PyTorch的scaled_dot_product_attention验证。
原文摘要 · Abstract (English)
We derive four memory-optimal inference artifacts for transformer attention using the Mathematics of Arrays (MoA), each following directly from the forward-pass Denotational Normal Form (DNF) of with the query-row index fixed to the current decode step. The artifacts are: (1)~a single-query decode DNF in which the $ψ$-reduction eliminates the $K^\top$ buffer algebraically, achieving $(d_k + nd_k+ nd_v+ d_v)\times4\,{B}$ Dynamic Random Access Memory (DRAM) traffic result numerically verified to $\|{err}\|_\leq2\times10^{-7}$; (2)~a C/OpenACC Graphics Processing Unit (GPU) kernel with Operational Normal Form (ONF) stride arithmetic and hardware-coalesced memory access, verified to $\|\mathrm{err}\|_\infty=0$ (exact IEEE-754 floating-point arithmetic); (3)~a multi-step KV-cache with $O(d_k+d_v)$ per-step append via MoA concatenation $\#$; and (4)~Grouped-Query Attention (GQA) and Multi-Query Attention (MQA) derived via $ψ$-selection, achieving a proven $\frac {h_q} { h_{kv} }$ reduction in KV traffic. All programs are verified against PyTorch scaled_dot_product_attention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。