用神经压缩技术提升脑影像解码效果，还能当降噪器用。
BrainCodec: Neural fMRI codec for the decoding of cognitive brain states
- 借鉴音频压缩思路，设计神经编码器压缩fMRI数据
- 在心理状态解码任务中性能超越已有方法
- 可解释的潜在表示，适合神经科学与医疗影像研究
近期深度学习利用大数据取得显著进展，体现在使用fMRI数据解码心理状态的应用中。然而，fMRI数据集规模仍较小，且固有的低信噪比（SNR）进一步加剧了挑战。为此，我们采用压缩技术作为fMRI数据的预处理步骤。提出BrainCodec，一种受神经音频编码器启发的新型fMRI编码器。评估了BrainCodec在心理状态解码中的压缩能力，结果表明其性能优于先前方法。此外，分析了通过BrainCodec获得的潜在表示，揭示了任务态与静息态fMRI之间的异同，突显了其可解释性。还证明，使用BrainCodec重建的fMRI图像能提高脑活动可见性，实现更高信噪比，表明其具有作为新型去噪方法的潜力。本研究显示，BrainCodec不仅提升了性能，还为神经科学提供了新的分析可能。代码、数据集和模型权重已公开于https://github.com/amano-k-lab/BrainCodec。
原文摘要 · Abstract (English)
Recently, leveraging big data in deep learning has led to significant performance improvements, as confirmed in applications like mental state decoding using fMRI data. However, fMRI datasets remain relatively small in scale, and the inherent issue of low signal-to-noise ratios (SNR) in fMRI data further exacerbates these challenges. To address this, we apply compression techniques as a preprocessing step for fMRI data. We propose BrainCodec, a novel fMRI codec inspired by the neural audio codec. We evaluated BrainCodec's compression capability in mental state decoding, demonstrating further improvements over previous methods. Furthermore, we analyzed the latent representations obtained through BrainCodec, elucidating the similarities and differences between task and resting state fMRI, highlighting the interpretability of BrainCodec. Additionally, we demonstrated that fMRI reconstructions using BrainCodec can enhance the visibility of brain activity by achieving higher SNR, suggesting its potential as a novel denoising method. Our study shows that BrainCodec not only enhances performance over previous methods but also offers new analytical possibilities for neuroscience. Our codes, dataset, and model weights are available at https://github.com/amano-k-lab/BrainCodec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。