AXELRAM让注意力计算直接用量化索引，省去反量化，大幅降低能耗。
AXELRAM: Quantize Once, Never Dequantize
- 设计固定码本，写入时变换，读取时查表，避免反变换和乘法运算
- 减少每查询乘法102.4倍，实验发现部分模型会出现50以上困惑度飙升
- 通过一次校准选择符号模式，零硬件成本解决稳定性问题
我们提出AXELRAM，一种智能的SRAM宏架构，可在不进行反量化的情况下，直接从量化后的键值缓存索引计算注意力得分。核心是设计阶段固定的码本：基于正交变换的量化将每个坐标的分布集中到N(0,1/d)，因此最优量化器仅依赖于维度d和位宽b，与输入数据无关。采用非对称路径设计——写入时变换，读取时查表且无逆变换——使每查询乘法减少102.4倍（数学恒等式）。通过多种子评估（10种子×3模型），我们发现符号模式敏感性会导致某些模型（Qwen2.5-3B）出现灾难性困惑度激增（Delta > 50），而其他模型（LLaMA-3.1-8B）则完全稳定。这一现象将SpinQuant在权重量化中的旋转变异性扩展至KV缓存域，影响更为严重。我们追溯根源为层间归一化异质性，并提出一种无需梯度的符号模式选择方法（200候选，8个校准样本，一次性完成），可零硬件开销消除灾难性峰值。所有源代码已公开于https://github.com/Axelidea/AXELRAM。
原文摘要 · Abstract (English)
We propose AXELRAM, a smart SRAM macro architecture that computes attention scores directly from quantized KV cache indices without dequantization. The key enabler is a design-time fixed codebook: orthogonal-transform-based quantization concentrates each coordinate's distribution to N(0,1/d), so the optimal quantizer depends only on dimension d and bit-width b, not on input data. The asymmetric path design -- transform on write, table-lookup on read with no inverse transform -- reduces per-query multiplications by 102.4x (a mathematical identity). Through multi-seed evaluation (10 seeds x 3 models), we discover that sign pattern sensitivity causes catastrophic PPL spikes (Delta > 50) on certain models (Qwen2.5-3B), while others (LLaMA-3.1-8B) are fully stable. This phenomenon extends SpinQuant's observation of rotation variance in weight quantization to the KV cache domain, where the effect is qualitatively more severe. We trace the root cause to layer-wise norm heterogeneity and propose a gradient-free sign pattern selection (200 candidates, 8 calibration samples, one-time) that eliminates catastrophic spikes with zero additional hardware cost. All source code is available at https://github.com/Axelidea/AXELRAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。