用神经网络可解释性图谱,找到虚拟现实晕动症时脑电的稳定特征。
Uncovering Patterns of Brain Activity from EEG Data Consistently Associated with Cybersickness Using Neural Network Interpretability Maps
- 设计自适应训练框架,提升小样本高变异脑电数据分类能力
- 多次实验一致发现顶区与颞区脑电模式与晕动症相关
- 提供可视化工具,助力实时晕动症检测系统开发
虚拟现实(VR)中的晕动症严重影响用户体验。为实现准确且实时的分类,需从脑电(EEG)数据中提取有效信号。但EEG数据量小、个体差异大,建模困难。本文提出一种基于神经网络的自适应训练与可解释性分析框架,结合卷积神经网络与变换器模型,利用集成梯度和类别激活映射生成可解释性图谱。在12次运行中,使用多个随机种子与三种模型,均一致识别出特定头皮位置(额顶区与颞区)及时间窗内脑电信号对晕动症判断最具判别力。该结果揭示了以往研究中隐含的脑活动模式,可作为未来实时分类的标记特征。代码已公开,支持跨架构特征解析。
原文摘要 · Abstract (English)
Cybersickness poses a serious challenge for users of virtual reality (VR) technology. Consequently, there has been significant effort to track its occurrence during VR use with passive measures like brain activity recorded through electroencephalogram (EEG). To classify cybersickness accurately, including in real time, machine learning algorithms which can extract meaningful signals from the rest of the brain data will be required. However, EEG datasets are typically very small and very high in variability between participants, which makes building effective models extremely challenging. To address these concerns, we first introduce a framework for neural networks which has subject-adaptive training with calibration and interpretation for classification given limited and imbalanced EEG data. Which features the models determine are most useful can be visualized by plotting interpretability maps from integrated gradients and class activation. The framework is demonstrated here with convolutional neural networks and transformer models. Using a set of brain data recorded with EEG while participants viewed a stimulus in VR designed to elicit cybersickness, we show which spatio-temporal EEG features (from electrodes and time steps) were most important for discomfort classification. Across 12 runs of our framework with three different neural networks over multiple random seeds, the models consistently pointed to the same scalp locations as having patterns of brain data that were the most helpful in determining whether or not a sample of EEG data belonged to someone who was experiencing cybersickness. These results help clarify a hidden pattern in other related research and can be used as tagged features for better real-time cybersickness classification with EEG in the future. We provide our code at [anonymized] to enable feature interpretation across different neural network architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。