arXiv:2604.19907cs.CV2026-04中稿 · ECCV被引 1

让3D场景生成一步到位,告别反复审查,效率与质量双提升。

SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation

  • 训练模型一次性生成完整工具调用流程,不再逐步审查。
  • 在相同时间内生成场景质量超越现有方法,且运行更快。
  • 适合需要高效生成高质量3D场景的开发者和设计师。

近期基于智能体的3D场景生成框架通过整合异构生成与编辑工具,显著提升了场景的真实感与多样性。这些工具由现成的大语言模型(LLM)进行调度,通常采用执行-审查-反思的循环机制:每一步执行工具后,渲染中间结果供审查,再决定下一步操作。然而该设计存在两大缺陷:一是后续步骤的工具选择与参数配置依赖启发式规则,易导致次优流程、冗余调用、质量下降和耗时增加;二是每步后需渲染并审查中间结果,引入额外延迟。为此,我们提出SceneOrchestra,一个可训练的调度框架,优化工具调用流程并移除逐步审查机制,从而提升效率与输出质量。SceneOrchestra包含调度器与判别器,采用两阶段训练策略:第一阶段,调度器学习上下文感知的工具选择与完整工具调用轨迹生成能力,判别器则训练以评估多条轨迹的质量,从中选出最优;第二阶段进行交替训练,判别器适应调度器轨迹分布,并将其判别能力回传给调度器。推理时仅使用调度器生成并执行完整的工具调用轨迹,无需判别器。大量实验表明,本方法在场景质量上达到当前最优水平。

原文摘要 · Abstract (English)

Recent agentic frameworks for 3D scene synthesis have advanced realism and diversity by integrating heterogeneous generation and editing tools. These tools are organized into workflows orchestrated by an off-the-shelf LLM. Current approaches typically adopt an execute-review-reflect loop: at each step, the orchestrator executes a tool, renders intermediate results for review, and then decides on the tool and its parameters for the next step. However, this design has two key limitations. First, next-step tool selection and parameter configuration are driven by heuristic rules, which can lead to suboptimal execution flows, unnecessary tool invocations, degraded output quality, and increased runtime. Second, rendering and reviewing intermediate results after each step introduces additional latency. To address these issues, we propose SceneOrchestra, a trainable orchestration framework that optimizes the tool-call execution flow and eliminates the step-by-step review loop, improving both efficiency and output quality. SceneOrchestra consists of an orchestrator and a discriminator, which we fine-tune with a two-phase training strategy. In the first phase, the orchestrator learns context-aware tool selection and complete tool-call trajectory generation, while the discriminator is trained to assess the quality of full trajectories, enabling it to select the best trajectory from multiple candidates. In the second phase, we perform interleaved training, where the discriminator adapts to the orchestrator's evolving trajectory distribution and distills its discriminative capability back into the orchestrator. At inference, we only use the orchestrator to generate and execute full tool-call trajectories from instructions, without requiring the discriminator. Extensive experiments show that our method achieves state-of-the-art scene quality compared to previous work.

3D生成智能体工具调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。