用光子计算加速Transformer，提升长序列处理效率
PGKET: A Photonic Gaussian Kernel Enhanced Transformer
- 基于光子干涉与叠加并行计算注意力得分
- 在MedMNIST v2和CIFAR-10上超越部分先进Transformer
- 适合光子计算与高效模型融合的研究者
自注意力机制虽能有效提取关键信息，但在处理长序列时效率低下。为此，本文提出光子高斯核增强变压器（PGKET），其核心为光子高斯核自注意力机制（PGKSAM）。该机制利用光子干涉与叠加原理，并行处理多输入以计算光子高斯核自注意力分数（PGKSAS）。实验表明，PGKET在MedMNIST v2和CIFAR-10的多分类任务中表现优于部分当前最优的Transformer模型，有望推动光子计算与机器学习的深度融合，提升复杂任务性能并加快收敛速度。
原文摘要 · Abstract (English)
Self-Attention Mechanisms (SAMs) enhance model performance by extracting key information but are inefficient when dealing with long sequences. To this end, a photonic Gaussian Kernel Enhanced Transformer (PGKET) is proposed, based on the Photonic Gaussian Kernel Self-Attention Mechanism (PGKSAM). The PGKSAM calculates the Photonic Gaussian Kernel Self-Attention Score (PGKSAS) using photon interferometry and superposition to process multiple inputs in parallel. Experimental results show that PGKET outperforms some state-of-the-art transformers in multi-classification tasks on MedMNIST v2 and CIFAR-10, and is expected to improve performance in complex tasks and accelerate the convergence of Photonic Computing (PC) and machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。