用可执行代码生成3D模型,实现更精准可控的编辑。
3D-CoS: A New 3D Reconstruction Paradigm Based on VLM Code Synthesis

- 将3D模型构建为可运行的Blender代码,程序化且可解释。
- 代码编辑比点云编辑更保真,未修改区域也更稳定。
- 适合需要精细控制的3D内容创作者或自动化建模研究者。
现有3D重建与编辑系统多基于NeRF、点云或网格等低层表示,虽能实现高保真渲染,但难以程序化控制。本文提出并系统评估一种新范式——3D代码合成(3D-CoS),将3D资产构造为可执行的Blender代码,实现程序化与可解释性。我们统一评估了主流开源与闭源视觉语言模型(VLM)在代码驱动重建中的表现。进一步设计了结构化代码生成工作流,包括蓝图规划、基于Blender API文档的检索增强生成(RAG)、少量几何示例引导及部件级智能体流程。通过局部文本驱动修改实验,对比代码与点云基编辑基线,结果表明代码表示具有更强的可控性与局部性,显著提升编辑保真度,并更好保留未编辑区域。本研究揭示了该范式的潜力,厘清了当前VLM在程序化3D建模中的能力边界,强调代码合成是可编辑3D重建的重要方向。
原文摘要 · Abstract (English)
Most recent 3D reconstruction and editing systems operate on implicit and explicit representations such as NeRF, point clouds, or meshes. While these representations enable high-fidelity rendering, they are fundamentally low-level and hard to control programmatically. In contrast, we propose and systematically evaluate a new 3D reconstruction paradigm, 3D Code Synthesis (3D-CoS), where 3D assets are constructed as executable Blender code, a programmatic and interpretable medium. To assess how well current VLMs can use code to represent 3D objects, we evaluate representative open-source and closed-source VLMs in code-based reconstruction under a unified protocol. We further introduce a suite of structured code-synthesis workflows, including blueprint-based planning, Retrieval-Augmented Generation (RAG) over Blender API documentation, few-shot geometric demonstrations, and a component-level Agent workflow for part-wise code generation. To demonstrate the unique advantages of this representation, we further evaluate localized text-driven modifications and compare our code-based edits with a point-cloud-based 3D editing baseline. Our study shows that code as a 3D representation offers strong controllability and locality, yielding stronger edit fidelity and better preservation of unedited regions in our targeted editing evaluation. Our work also analyzes the potential of this paradigm, delineates the current capability frontier of VLMs for programmatic 3D modeling, and highlights code synthesis as a promising direction for editable 3D reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。