将softmax注意力映射到量子概率空间,实现精确的量子等价计算。
A Quantum Roadmap for Softmax Attention: Exact Born-Rule Analogs for Softmax Attention on the Probability Simplex
- 用量子测量原理重构注意力分数,通过哈达玛测试实现精确计算。
- 温度参数对应测量次数,有限值下可生成精确稀疏注意力零点。
- 适合对量子机器学习与注意力机制交叉研究感兴趣的读者。
注意力机制是现代AI模型(如Transformer)的核心。在输入输出受限于概率单纯形(和为1)的场景中,softmax注意力存在精确的逐分量量子实现:注意力分数是幅度编码输入的块编码投影的哈达玛测试统计量。指数型softmax是基于玻恩规则测量生成的余弦平方族的内部,其边界在有限参数值下表现出精确的稀疏注意力(零值)。温度参数对应重复测量次数,后选择测量回合可精确实现离散反温度。值聚合是一个确定性列加载通道,使列随机值矩阵得到拉伸。门控残差是单个辅助量子比特的准备角,加法单位对应π/2混合角。所有可学习参数均为旋转门角度。组合层在无限采样极限下精确,每注意力分数仅需一次测量-重载步骤;全相干变体在无限深度极限下以ε近似通过量子奇异值变换实现。代数核心已在Lean 4中经机器验证。
原文摘要 · Abstract (English)
The attention mechanism forms the foundation of many modern AI models such as the Transformer. In one subclass of problems where attention is used, inputs and outputs are bound to the probability simplex so that all outputs sum to one. In this setting, softmax attention admits an exact, component-by-component quantum realization. Attention scores are Hadamard-test statistics on block-encoded projections of amplitude-encoded inputs. The exponential softmax is the interior of a cosine-squared family generated by Born-rule measurement under an exact bijection, whose boundary expresses sparse attention with exact zeros at finite parameter values. The softmax temperature is a repetition count where post-selected measurement rounds realize discretized inverse temperature exactly. Value aggregation is a deterministic column-loading channel that dilates the column-stochastic value matrix. The gated residual is the preparation angle of a single ancilla, with the additive identity at a mixing angle of π/2. Every learnable parameter is a rotation-gate angle. The composed layer is exact in the infinite-shot limit with one measure-and-reload step per attention score; a fully-coherent variant is ε-approximate via quantum singular value transformation in the infinite depth limit. The algebraic core is machine-checked in Lean 4.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。