arXiv:2606.26964cs.AIcs.CV2026-06

让摄像机先看再动,根据叙事意图智能规划视觉焦点。

Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds

论文配图:Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds
图 1 · 摘自论文原文
  • 先定观察目标再生成镜头运动,分离感知与动作
  • 在50个故事、1585个镜头上提升感知与叙事一致性
  • 适合影视生成、虚拟导演等需要精准视觉控制的场景

随着具身智能和世界模型在动态3D环境中应用增多,视觉感知需从被动解读观测转向主动决策观察内容。本文研究动态3D故事世界中的摄像机规划问题,要求摄像机不仅生成平滑运动,还需在移动前决定应获取哪些视觉证据。为此提出「叙事锚定世界视觉注意力」机制,将摄像机视为具身观察者,依据叙事意图与三维物理约束,自主决定观察内容、组合方式及注意力转移。提出Look-Before-Move框架:首先构建语义观察契约,将导演意图转化为可执行视觉约束;接着通过蒙特卡洛视角搜索,寻找符合叙事且几何可行的视角;最后通过语义轨迹锚定,将选中视角连接为连续、避障且时间连贯的相机运动。进一步基于StoryBlender构建动态3D故事世界基准,涵盖50个故事、457个场景、1585个镜头,包含动画角色、语义场景配置与可执行3D环境。实验表明,该框架在主体感知、意图一致性和轨迹质量上优于代表性基线,验证了先组织视觉注意力再生成运动的重要性。

原文摘要 · Abstract (English)

As embodied AI and world models increasingly operate in dynamic 3D environments, visual perception must move beyond passively interpreting given observations toward actively deciding what to observe. We study this problem through camera planning in dynamic 3D story worlds, where the camera must not only generate smooth motion, but also decide what visual evidence should be acquired before it moves. We formulate this capability as Narrative-Grounded World Visual Attention, where the camera acts as an embodied observer that determines what to observe, how to compose the observation, and how to shift attention over time under narrative intent and physical 3D constraints. To realize this capability, we propose Look-Before-Move, a camera planning framework that separates observation specification from motion execution. It first builds a Semantic Observation Contract to convert directorial intent into executable visual constraints, then performs Monte Carlo Viewpoint Search to find narrative-compliant and geometrically feasible viewpoints, and finally applies Semantic Trajectory Grounding to connect selected viewpoints into continuous, collision-aware, and temporally coherent camera motion. We further construct a dynamic 3D Story World Benchmark based on StoryBlender, covering 50 stories, 457 scenes, and 1585 shots with animated characters, semantic scene configurations, and executable 3D environments. Experiments show that our framework improves subject perception, intent consistency, and trajectory quality over representative baselines, demonstrating the importance of organizing visual attention before generating camera motion.

视觉注意力3D生成叙事生成摄像机规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。