提出自由能混合器，实现通道自适应注意力读取
Free Energy Mixer
- 用自由能机制对键值进行价值驱动的通道倾斜读取
- 在相同复杂度下，性能优于主流注意力模型
- 可无缝接入标准与线性注意力，适用于多模态任务
标准注意力无损存储键值，但通过每头凸平均读取，限制了通道选择。本文提出自由能混合器（FEM）：基于自由能（log-sum-exp）的读取方式，以值为导向对快速先验（如查询/键生成）的索引施加对数线性倾斜。不同于提升 $(q,k)$ 评分分布的方法，FEM 将其视为先验，得到价值感知的后验读取，在不变复杂度下，随可学习逆温度增加，平滑从平均过渡到逐通道选择，同时保持并行性与原始渐近复杂度（softmax 为 $O(T^2)$；可线性化变体为 $O(T)$）。我们设计了两级门控 FEM，可即插即用于标准注意力、线性注意力、线性 RNN 和 SSM。在自然语言处理、视觉和时间序列任务中，参数量匹配时持续超越强基线。
原文摘要 · Abstract (English)
Standard attention stores keys/values losslessly but reads them via a per-head convex average, blocking channel-wise selection. We propose the Free Energy Mixer (FEM): a free-energy (log-sum-exp) read that applies a value-driven, per-channel log-linear tilt to a fast prior (e.g., from queries/keys in standard attention) over indices. Unlike methods that attempt to improve and enrich the $(q,k)$ scoring distribution, FEM treats it as a prior and yields a value-aware posterior read at unchanged complexity, smoothly moving from averaging to per-channel selection as the learnable inverse temperature increases, while still preserving parallelism and the original asymptotic complexity ($O(T^2)$ for softmax; $O(T)$ for linearizable variants). We instantiate a two-level gated FEM that is plug-and-play with standard and linear attention, linear RNNs and SSMs. It consistently outperforms strong baselines on NLP, vision, and time-series at matched parameter budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。