让脑电基础模型更快更轻，本地运行也不丢精度。
ZIPBrain: Can EEG Foundation Models Be Faster, Locally Deployable, but Accurate?
- 用冗余感知方法压缩脑电token，减少计算量。
- 平均提速32.7%，最高达41.8%，精度反而提升1.3%-10.5%。
- 无需重训练，可直接插入现有模型，适合临床实时应用。
本文探究脑电图（EEG）基础模型能否在不损失精度的前提下实现更快、更易本地部署。尽管EEG基础模型具备强泛化能力，但其计算开销随输入长度呈二次增长，难以在资源受限场景下应用，尤其影响实时临床监测。由于EEG信噪比低，大量token具有冗余性，可压缩而几乎不影响精度。为此，提出ZIPBrain——一种新型冗余感知的脑电token池化模块，将输入序列分为冗余与独特两类，将每个冗余项与其最相似的独特项合并。该模块无需训练、可即插即用，嵌入标准Transformer编码器后仅带来极小额外开销。跨多个EEG基础模型的实验表明，ZIPBrain展现强大通用性：相比基线平均提升1.3%-10.5%，推理时间减少32.7%（使用CUDA Graph最高达41.8%）。
原文摘要 · Abstract (English)
This work investigates whether Electroencephalograph (EEG) foundation models (EFMs) can be made faster and locally deployable without sacrificing accuracy. EEG foundation models are a major trend, offering strong general-purpose representations. However, their computational burden grows quadratically with input length, hindering deployment on resource-constrained scenario, particularly for real-time clinical monitoring. EEG's low SNR further suggests many of these tokens are redundant and compressible with little accuracy cost. We propose ZIPBrain, a novel redundancy-aware EEG token pooling module that leverages this low-SNR characteristic to reduce token count. Given a token sequence, ZIPBrain partitions tokens into redundant and unique groups, then merges each redundant token with its most similar counterpart in the unique group. Furthermore, ZIPBrain serves as a training-free, plug-and-play module that seamlessly integrates into standard Transformer encoders with negligible computational overhead. Extensive experiments across multiple EEG foundation models show ZIPBrain's strong versatility, achieving 1.3%-10.5% average improvement over baselines, while reducing wall-clock inference time by 32.7% (up to 41.8% with CUDA Graph) compared to the original EEG foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。