arXiv:2606.01057cs.CVcs.AI2026-06被引 7

评测视觉语言模型生成3D代码的能力,推动自动化3D建模发展。

3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

论文配图:3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code
图 1 · 摘自论文原文
  • 构建基准测试体系,评估12个视觉语言模型将图文转为3D代码的性能。
  • 发现代码生成失败主因是接口不匹配,成功结果仍存在几何部件分离问题。
  • 提供可公开访问的评测平台与数据集,适合研究生成式AI与3D建模交叉方向者。

通过代码实现程序化3D建模正成为一种灵活范式,能生成确定性、可直接运行且易于编辑的资产,这是神经3D生成器所缺乏的。然而,编写此类内容需要深入掌握3D软件API、参数化设计及代码级几何推理知识。本文提出3DCodeBench,一个系统性基准,用于评估视觉语言模型(VLM)代理在3D建模软件中进行程序化3D生成的能力。具体而言,3DCodeBench评估12个先进VLM将文本和图像参考转化为3D建模软件可用的程序化代码的有效性。鉴于自动度量可能无法完全捕捉3D形状的感知质量,我们构建了基于成对人类偏好评分的3DCodeArena评测平台。大量实验表明:(1)失败主要源于API不匹配,而成功渲染仍存在几何组件断开或悬浮问题;(2)测试时扩展(如更高的思维预算和多轮优化)整体提升了性能。研究强调了高质量程序化编码数据对提升商业级VLM的重要性,同时指出有效3D建模需具备高保真反馈的执行环境以支持迭代优化。我们发布了3DCodeBench,包含经筛选的大型多模态(文本/图像)提示数据集、程序化代码、3D物体三元组、评估协议及公开的3DCodeArena平台,作为探索基于VLM的程序化3D建模者的基础工具包。

原文摘要 · Abstract (English)

Procedural 3D modeling through code is emerging as a versatile paradigm, offering deterministic, engine-ready, and precisely editable assets that neural 3D generators inherently lack. Authoring such procedural content, however, demands deep expertise in 3D software APIs, parametric design, and code-level geometric reasoning. In this paper, we propose 3DCodeBench, a systematic benchmark for evaluating vision-language model (VLM) agents for procedural 3D generation in 3D modeling software. Specifically, 3DCodeBench evaluates how effectively 12 advanced VLMs can serve as procedural 3D modelers by translating text and image references into procedural code for 3D modeling software. Recognizing that automated metrics may not fully capture the perceptual quality of 3D shapes, we build 3DCodeArena, a ranking platform based on pairwise human preferences over generated 3D outputs. From extensive evaluations and results, we observe that: (1) Failures mostly arise from API mismatches, while successful renders still suffer from disconnected or floating 3D geometric components. (2) Test-time scaling, such as higher thinking budgets and multi-turn refinement, improves performance overall. Our findings highlight a critical need for high-quality procedural coding data to advance commercial VLMs. Furthermore, effective procedural 3D modeling requires a robust execution environment that provides high-fidelity feedback for iterative refinement. We release 3DCodeBench, including the curated large-scale dataset of multimodal (text/image) prompts, procedural code, 3D object triplets, evaluation protocol, and the public 3DCodeArena platform as a foundational toolkit for exploring VLM-based procedural 3D modelers.

3D建模视觉语言模型程序化生成人机评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。