从无标签EEG数据构建预训练数据集，提升Transformer模型训练效率与准确率。
Designing Pre-training Datasets from Unlabeled Data for EEG Classification with Transformers
- 基于无标签EEG数据设计多组有标签预训练数据集
- 预训练后微调时间减少超50%，准确率提升至92.16%
- 适合缺乏标注数据的脑电分类研究者使用
Transformer神经网络需要大量标注数据才能有效训练，但脑电图（EEG）领域因医疗专家标注成本高，标注数据往往稀缺。因此，需先利用无标签数据进行自监督预训练。本文提出一种从无标签EEG数据中设计多个有标签数据集的方法，用于预训练Transformer模型以学习脑电信号表征。我们在Temple University Seizure Detection Corpus数据集上，针对癫痫发作预测任务，采用多通道视觉变换器进行了测试。结果表明：1）使用该方法预训练的模型显著缩短了微调时间，相比非预训练模型，微调时长减少超过50%；2）预训练模型准确率从90.93%提升至92.16%，AUC由0.9648增至0.9702。
原文摘要 · Abstract (English)
Transformer neural networks require a large amount of labeled data to train effectively. Such data is often scarce in electroencephalography, as annotations made by medical experts are costly. This is why self-supervised training, using unlabeled data, has to be performed beforehand. In this paper, we present a way to design several labeled datasets from unlabeled electroencephalogram (EEG) data. These can then be used to pre-train transformers to learn representations of EEG signals. We tested this method on an epileptic seizure forecasting task on the Temple University Seizure Detection Corpus using a Multi-channel Vision Transformer. Our results suggest that 1) Models pre-trained using our approach demonstrate significantly faster training times, reducing fine-tuning duration by more than 50% for the specific task, and 2) Pre-trained models exhibit improved accuracy, with an increase from 90.93% to 92.16%, as well as a higher AUC, rising from 0.9648 to 0.9702 when compared to non-pre-trained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。