优化激活函数提升脑电转文本模型性能，多项式函数有效但有局限。
On the Role of Activation Functions in EEG-To-Text Decoder
- 测试多种激活函数，发现高阶多项式可提升性能
- 可学习的3次多项式在1-gram上优于固定版本
- Leaky ReLU在2-gram及以上指标表现更优
近年来，跨学科研究探索了神经科学在信息检索中的应用潜力。早期研究多基于fMRI数据，但因其不适用于真实场景，研究转向使用EEG数据。本文通过优化神经网络性能，改进首次尝试的脑电转文本模型。我们测试了多种激活函数并比较其表现。结果表明，引入高阶多项式激活函数可在不改变模型结构的前提下提升性能。可学习的3次多项式在1-gram评估中优于固定3次多项式，但在2-gram及以上评估中表现不佳；相比之下，Leaky ReLU在高阶词元上优于基线模型。
原文摘要 · Abstract (English)
In recent years, much interdisciplinary research has been conducted exploring potential use cases of neuroscience to advance the field of information retrieval. Initial research concentrated on the use of fMRI data, but fMRI was deemed to be not suitable for real-world applications, and soon, research shifted towards using EEG data. In this paper, we try to improve the original performance of a first attempt at generating text using EEG by focusing on the less explored area of optimising neural network performance. We test a set of different activation functions and compare their performance. Our results show that introducing a higher degree polynomial activation function can enhance model performance without changing the model architecture. We also show that the learnable 3rd-degree activation function performs better on the 1-gram evaluation compared to a 3rd-degree non-learnable function. However, when evaluating the model on 2-grams and above, the polynomial function lacks in performance, whilst the leaky ReLU activation function outperforms the baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。