用少量数据快速适配新受试者，实现脑信号到视频的跨人重建
MindCross: Fast New Subject Adaptation with Limited Data for Cross-subject Video Reconstruction from Brain Signals
- 分设专属与共享编码器，分离个体特异与共性信息
- 仅需1个模型即可完成跨受试者视频重建，适配速度快
- 适合脑机接口、神经解码领域研究者使用
从脑信号重构视频是一项重要的脑解码任务。现有框架多为受试者依赖型，需大量个体脑数据，而脑-视频数据采集成本高，导致数据严重稀缺。尽管已有跨受试者方法，但常过度关注不变特征，忽略个体差异，导致适应过程缓慢。为此，我们提出MindCross，一种新型跨受试者框架。该框架包含多个专属编码器和一个共享编码器，分别提取受试者特异性与共性信息；同时引入Top-K协作模块，利用先前受试者编码器的知识提升新受试者的解码性能。在fMRI/EEG-to-video基准上的大量实验表明，MindCross在跨受试者重建与新受试者快速适应方面均表现出高效性，且仅需一个模型即可完成。
原文摘要 · Abstract (English)
Reconstructing video from brain signals is an important brain decoding task. Existing brain decoding frameworks are primarily built on a subject-dependent paradigm, which requires large amounts of brain data for each subject. However, the expensive cost of collecting brain-video data causes severe data scarcity. Although some cross-subject methods being introduced, they often overfocus with subject-invariant information while neglecting subject-specific information, resulting in slow fine-tune-based adaptation strategy. To achieve fast and data-efficient new subject adaptation, we propose MindCross, a novel cross-subject framework. MindCross's N specific encoders and one shared encoder are designed to extract subject-specific and subject-invariant information, respectively. Additionally, a Top-K collaboration module is adopted to enhance new subject decoding with the knowledge learned from previous subjects' encoders. Extensive experiments on fMRI/EEG-to-video benchmarks demonstrate MindCross's efficacy and efficiency of cross-subject decoding and new subject adaptation using only one model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。