用大模型和3D生成技术让梦境可视化，实现沉浸式重历梦中体验。
DreamLLM-3D: Affective Dream Reliving using Large Language Model and 3D Generative AI
- 大模型分析口述梦话，提取人物、物体与情绪信息。
- 将梦中实体转为动态3D点云，情绪影响颜色与音景。
- 提出人机协同梦工坊范式，适合艺术与心理探索者。
我们提出DreamLLM-3D，一个用于沉浸式梦境重历艺术装置的多模态复合人工智能系统。该系统通过整合大型语言模型（LLM）与文本到3D生成AI，实现对口头梦境报告的自动化内容分析，以支持沉浸式梦境重历。LLM负责解析梦话，识别关键梦中实体（人物与物体）、社会互动及梦境情感。提取出的实体被转化为动态3D点云，其情绪数据影响虚拟梦境环境的颜色与声音景观。此外，我们提出一种体验式的人工智能-梦工坊混合范式。该系统与范式有望提升梦境重历的情感沉浸感，促进个人洞察力与创造力。
原文摘要 · Abstract (English)
We present DreamLLM-3D, a composite multimodal AI system behind an immersive art installation for dream re-experiencing. It enables automated dream content analysis for immersive dream-reliving, by integrating a Large Language Model (LLM) with text-to-3D Generative AI. The LLM processes voiced dream reports to identify key dream entities (characters and objects), social interaction, and dream sentiment. The extracted entities are visualized as dynamic 3D point clouds, with emotional data influencing the color and soundscapes of the virtual dream environment. Additionally, we propose an experiential AI-Dreamworker Hybrid paradigm. Our system and paradigm could potentially facilitate a more emotionally engaging dream-reliving experience, enhancing personal insights and creativity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。