arXiv:2507.21893cs.CV2025-07被引 1

多模态叙事生成框架,同步产出故事、场景图与情感音效。

Aether Weaver: Multimodal Affective Narrative Co-Generation with Dynamic Scene Graphs

  • 用统一机制同时生成文本、动态场景图、视觉与情感音效。
  • 相比串行流程,故事深度、画面质量与情感共鸣显著提升。
  • 适合创意原型设计与沉浸式叙事应用,如游戏与影视开发。

我们提出Aether Weaver,一种新型集成式多模态叙事协同生成框架,突破了传统文本到视觉的串行流程局限。系统通过紧密耦合的协同生成机制,同步合成叙事文本、动态场景图、视觉画面与情感音景。核心组件包括:作为叙事生成器的大语言模型Narrator,负责生成文本与多模态提示;作为场景图管理器的Director,分析文本并构建、维护故事世界的结构化表示,保障时空与关系一致性;叙事弧控制器(Narrative Arc Controller)调控高层故事结构,影响多模态情感一致性;情感基调映射器(Affective Tone Mapper)确保各模态间情感表达协调。在涵盖多种类型叙事提示的定性评估中,Aether Weaver相比串行基线方法,在叙事深度、视觉保真度和情感感染力方面均有显著提升。该框架为快速创意原型制作与沉浸式叙事体验提供了强大平台。

原文摘要 · Abstract (English)

We introduce Aether Weaver, a novel, integrated framework for multimodal narrative co-generation that overcomes limitations of sequential text-to-visual pipelines. Our system concurrently synthesizes textual narratives, dynamic scene graph representations, visual scenes, and affective soundscapes, driven by a tightly integrated, co-generation mechanism. At its core, the Narrator, a large language model, generates narrative text and multimodal prompts, while the Director acts as a dynamic scene graph manager, and analyzes the text to build and maintain a structured representation of the story's world, ensuring spatio-temporal and relational consistency for visual rendering and subsequent narrative generation. Additionally, a Narrative Arc Controller guides the high-level story structure, influencing multimodal affective consistency, further complemented by an Affective Tone Mapper that ensures congruent emotional expression across all modalities. Through qualitative evaluations on a diverse set of narrative prompts encompassing various genres, we demonstrate that Aether Weaver significantly enhances narrative depth, visual fidelity, and emotional resonance compared to cascaded baseline approaches. This integrated framework provides a robust platform for rapid creative prototyping and immersive storytelling experiences.

多模态生成叙事生成动态场景图情感一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。