提出共享分数的四元数自注意力，显著降低计算开销且保持性能。
Quaternion Self-Attention with Shared Scores
- 用四元数内积计算单一实值分数，共享注意力分布
- 推理速度提升44.3%（GPU）和58.1%（CPU）
- 适用于语音、视觉与自然语言处理任务
四元数神经网络通过将四个相关特征合并为单一实体，实现参数高效并建模多维依赖。然而，现有四元数自注意力分别计算各分量得分，并对每个分量独立进行Softmax操作，导致计算成本上升且各分量注意力分布可能发散。本文提出一种共享分数的四元数自注意力机制：利用四元数内积计算单一实值分数，并在所有分量间共享注意力分布。该方法将得分计算乘法减少75%，Softmax操作从四个降至一个。我们证明，当查询与键由引入分量预混合的四元数线性投影生成时，分量式与共享得分位于同一交互子空间，表明独立分量注意力主要对相同交互进行重参数化，而非扩展特征交互空间。在语音增强任务中，该方法在GPU上推理时间减少最多44.3%，在CPU上减少58.1%，且在视觉与自然语言处理任务中也表现出一致的速度提升趋势。
原文摘要 · Abstract (English)
Quaternion neural networks are parameter-efficient and model multidimensional dependencies by representing four related features as a single entity. However, existing quaternion self-attention computes component-wise scores and applies independent softmax operations to each component, which increases the computational cost and allows attention distributions to diverge across components. We propose a shared-score quaternion self-attention mechanism that computes a single real-valued score using the quaternion inner product and applies a shared attention distribution across all components. This reduces score-computation multiplications by 75% and the number of softmax operations from four to one. We prove that, when queries and keys are produced by quaternion linear projections that induce component pre-mixing, the component-wise and shared scores lie in the same interaction subspace, indicating that independent component-wise attention primarily re-parameterizes the same interactions rather than expanding the feature interaction space. In speech enhancement, our method reduces inference time by up to 44.3% on a GPU and 58.1% on a CPU while maintaining quality, with consistent trends across vision and natural language processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。