用fMRI信号解码梦境并生成连贯视频故事
Making Your Dreams A Reality: Decoding the Dreams into a Coherent Video Story from fMRI Signals
- 通过fMRI数据重建视觉感知，提取梦中图像
- 结合语言模型将梦境片段整合为完整叙事
- 首个尝试从神经信号生成连续梦境视频的框架
本文提出一种新框架，利用功能磁共振成像（fMRI）信号将梦境转化为连贯的视频叙事。研究聚焦于人类对梦境数世纪以来的好奇，借助近年来脑成像技术的发展，特别是fMRI，探索梦境的神经基础。该方法包含三个核心步骤：视觉感知重建、梦境意象解码和梦境故事整合。通过创新的fMRI分析与语言建模技术，旨在突破梦境研究的边界，深入理解睡眠中的视觉体验。研究团队构建了一个包含梦境及描述的数据集，用于评估框架的有效性。
原文摘要 · Abstract (English)
This paper studies the brave new idea for Multimedia community, and proposes a novel framework to convert dreams into coherent video narratives using fMRI data. Essentially, dreams have intrigued humanity for centuries, offering glimpses into our subconscious minds. Recent advancements in brain imaging, particularly functional magnetic resonance imaging (fMRI), have provided new ways to explore the neural basis of dreaming. By combining subjective dream experiences with objective neurophysiological data, we aim to understand the visual aspects of dreams and create complete video narratives. Our process involves three main steps: reconstructing visual perception, decoding dream imagery, and integrating dream stories. Using innovative techniques in fMRI analysis and language modeling, we seek to push the boundaries of dream research and gain deeper insights into visual experiences during sleep. This technical report introduces a novel approach to visually decoding dreams using fMRI signals and weaving dream visuals into narratives using language models. We gather a dataset of dreams along with descriptions to assess the effectiveness of our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。