arXiv:2604.10383cs.CV2026-04被引 2

用工具约束的LLM代理生成可执行的多角色动态场景,让故事自动变成视频。

Authoring for Living Worlds: Tool-Constrained LLM Agents for Executable Multi-Actor Scenarios

论文配图:Authoring for Living Worlds: Tool-Constrained LLM Agents for Executable Multi-Actor Scenarios
图 1 · 摘自论文原文
  • 引入工具层控制代理行为,确保每步操作符合世界状态
  • 80%生成场景可执行,事件匹配率达0.83 F1,优于随机水平
  • 适合想自动化创作动态叙事内容的研究者和开发者

为动态3D世界创作多角色剧情,要求叙事自由与模拟严谨并重。我们使用大语言模型(LLM)代理构建时空事件图(GEST),由仿真引擎确定性地执行为带逐帧空间、时间与语义真值的叙事视频。传统分阶段流水线完全失败:模型无视提示中的规则且无法跟踪动态世界状态。我们提出约束式工具层——导演与场景构建代理逐页探索世界能力,通过与仿真器状态校验的操作构建每幕场景,确保生成规范均合法。因种子文本源自已有场景图,可进行重构评估:代理仅凭文本重建原图,事件匹配达0.83 F1(参与者0.55,顺序0.77),远超随机基准(0.43)。端到端测试中,标准流水线50次尝试产出0个可执行规范;而我们的代理使用预算级模型,在25次中成功执行20次(80%),据我们所知首次完整发挥GEST的表达潜力。

原文摘要 · Abstract (English)

Authoring a multi-actor scenario for a living 3D world, where every action changes its state, and each action's validity depends on the state accumulated before it, demands the freedom of storytelling and the rigor of simulation at once. We author such scenarios with LLM agents, as Graphs of Events in Space and Time (GESTs) that a simulation engine executes deterministically into narrative videos with per-frame spatial, temporal, and semantic ground truth. A staged pipeline driving a flagship LLM, the standard design in video generation, failed outright: the model violates rules stated verbatim in its prompt, and cannot track the dynamic world state. We answer with a constraint-enforcing tool layer: our Director and Scene Builder agents explore the world's capabilities page by page and build every scene through operations checked against simulator state, so every specification they emit is valid by construction. Because we generate each seed text from an existing scenario graph, we can measure reconstruction: the agent authors its own graph from the text alone, yet matches the original at 0.83 F1 on its events, each action with its participants (0.55 for a random scenario of the same kind), and 0.77 on their ordering (0.43 random). End to end: the standard staged pipeline produced 0 executable specifications in 50 attempts; our agents, driving a budget model, execute 20 of 25 (80%), and are, to our knowledge, the first to exercise the full expressive capacity of GEST.

多角色生成工具约束场景生成可控叙事

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。