用单阶段扩散模型从动态脑电波还原图像，突破时间分辨率瓶颈。
Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI
- 直接建模连续fMRI信号，无需多阶段预处理
- 在高层次语义重建上超越现有模型表现
- 适合研究脑活动动态演化过程的神经科学工作者
脑信号到图像的解码近年来因生成式AI和高场强功能磁共振成像（fMRI）的发展而取得进展。然而，现有方法依赖复杂的多阶段流程与预处理步骤，通常会压缩脑信号的时间维度，限制了时间分辨的脑解码能力。本文提出Dynadiff（动态神经活动扩散模型），一种用于从动态演变的fMRI记录中重建图像的单阶段扩散模型。该方法具有三大贡献：首先，相比现有方法，训练过程更简化；其次，在时间分辨的fMRI信号上表现优于当前最优模型，尤其在高层语义图像重建指标上，同时在时间塌陷的预处理数据上仍保持竞争力；第三，该方法可精确刻画图像表征在脑活动中的演化过程。本工作为时间分辨的脑-图像解码奠定了基础。
原文摘要 · Abstract (English)
Brain-to-image decoding has been recently propelled by the progress in generative AI models and the availability of large ultra-high field functional Magnetic Resonance Imaging (fMRI). However, current approaches depend on complicated multi-stage pipelines and preprocessing steps that typically collapse the temporal dimension of brain recordings, thereby limiting time-resolved brain decoders. Here, we introduce Dynadiff (Dynamic Neural Activity Diffusion for Image Reconstruction), a new single-stage diffusion model designed for reconstructing images from dynamically evolving fMRI recordings. Our approach offers three main contributions. First, Dynadiff simplifies training as compared to existing approaches. Second, our model outperforms state-of-the-art models on time-resolved fMRI signals, especially on high-level semantic image reconstruction metrics, while remaining competitive on preprocessed fMRI data that collapse time. Third, this approach allows a precise characterization of the evolution of image representations in brain activity. Overall, this work lays the foundation for time-resolved brain-to-image decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。