arXiv:2606.06002cs.CV2026-06

用树搜索生成3D室内场景,避免错误累积。

Global-Local Monte Carlo Tree Search in Vision-Language Models for Text-to-3D Indoor Scene Generation

论文配图:Global-Local Monte Carlo Tree Search in Vision-Language Models for Text-to-3D Indoor Scene Generation
图 1 · 摘自论文原文
  • 分全局与局部树搜索,模拟人类布置房间的多轮尝试。
  • 仅需较少尝试即生成逼真场景,优于现有方法。
  • 适合需要高质量3D场景生成的研究者和开发者。

大型视觉语言模型在多种任务中表现出色,但针对文本到3D室内场景生成的研究仍较少。主要挑战在于现有方法采用链式思维的顺序决策机制,无法修正早期决策,导致错误传播。本文将该任务视为受空间与布局常识约束的规划问题,建模为包含全局与局部树的搜索问题。全局树中,物体逐个放置并探索多种尝试,问题空间以树形表示;为此提出分层场景表示与基于PRM的蒙特卡洛树搜索(PRM-guided MCTS),通过PRM剪枝无效分支,结合MCTS平衡探索与利用,以更少尝试获得最优解。局部树进一步将每个物体的放置分解为具体参数步骤。为保证场景外观一致性,使用预训练扩散图像生成模型预测所有物体的纹理。由于现有基准数据集规模与多样性有限,本文构建了新数据集3DTindo-bench,包含65种场景类型、3250条指令,涵盖多样尺寸、布局与风格,用于更全面评估先进模型。实验表明,本方法生成的3D场景更具真实感。

原文摘要 · Abstract (English)

Large Vision-Language Models have achieved significant reasoning performance in various tasks. However, there are few studies on text-to-3D indoor scene generation with LVLMs. The main challenge is that prevailing LVLM-based methods employ chain-of-thought sequential decision mechanisms that cannot revise earlier decisions, causing error propagation. In this paper, we consider the task as a planning problem constrained by spatial and layout commonsense. To solve this problem, we model it as a tree search problem with global and local trees, which differs from existing sequential decision-making approaches. In the global tree, we place each object iteratively and explore multiple attempts like humans furnishing a room, where the problem space is represented as a tree. To effectively search the tree, we propose a hierarchical scene representation and a PRM-guided MCTS method. This representation abstracts a scene into room level, region level, floor object level, and supported object level. The PRM-guided MCTS method uses the PRM to prune unnecessary branches and the MCTS algorithm to balance exploration and exploitation to get an optimal solution with fewer attempts. In the local tree, it further decomposes the placement of each object into finer sub-steps, including the specific placement parameters. To make the whole appearance of the scene consistent, we leverage pre-trained diffusion image generative models to predict textures for all the objects in the scene. As existing benchmarks for text-to-3D indoor scene generation remain limited in scale and diversity, we collect a new large-scale diverse dataset that contains 65 scene types and 3250 instructions with diverse sizes, layouts, and styles, named 3DTindo-bench, to better assess the capability of the state-of-the-art models. Our experiments show that our method generates more realistic 3D scenes than state-of-the-art methods.

3D生成树搜索视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。