arXiv:2510.22335cs.CVcs.AI2025-10

用分层自回归方法,从脑电数据更高效地还原图像。

Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction

  • 分层编码脑信号,按尺度逐步生成图像
  • 重建精度更高,速度比扩散模型快4.67倍
  • 适合脑机接口与认知神经科学研究

从功能性磁共振成像(fMRI)信号重建视觉刺激是机器学习与神经科学交叉的核心挑战。现有基于扩散的方法通常将fMRI活动映射为单一神经嵌入,全程使用静态引导,导致层次化神经信息丢失,且与图像生成阶段需求不匹配。为此,我们提出MindHier框架,一种基于尺度自回归建模的粗到细重建方法。该框架包含三个组件:分层fMRI编码器提取多层级神经嵌入,层次到层次对齐机制强制与CLIP特征逐层对应,以及尺度感知的粗到细神经引导策略,在匹配尺度注入嵌入。这些设计使MindHier成为高效且符合认知规律的替代方案,实现先生成全局语义再细化局部细节的层次化重建过程,类似人类视觉感知。在NSD数据集上的大量实验表明,MindHier在语义保真度上优于扩散基线,推理速度提升4.67倍,结果更稳定确定。

原文摘要 · Abstract (English)

Reconstructing visual stimuli from fMRI signals is a central challenge bridging machine learning and neuroscience. Recent diffusion-based methods typically map fMRI activity to a single neural embedding, using it as static guidance throughout the entire generation process. However, this fixed guidance collapses hierarchical neural information and is misaligned with the stage-dependent demands of image reconstruction. In response, we propose MindHier, a coarse-to-fine fMRI-to-image reconstruction framework built on scale-wise autoregressive modeling. MindHier introduces three components: a Hierarchical fMRI Encoder to extract multi-level neural embeddings, a Hierarchy-to-Hierarchy Alignment scheme to enforce layer-wise correspondence with CLIP features, and a Scale-Aware Coarse-to-Fine Neural Guidance strategy to inject these embeddings into autoregression at matching scales. These designs make MindHier an efficient and cognitively aligned alternative to diffusion-based methods by enabling a hierarchical reconstruction process that synthesizes global semantics before refining local details, akin to human visual perception. Extensive experiments on the NSD dataset show that MindHier achieves superior semantic fidelity, 4.67$\times$ faster inference, and more deterministic results than the diffusion-based baselines.

脑机接口图像生成自回归fMRI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。