用大模型生成可实时参数化编辑的3D模型,效率提升400倍。
Proc3D: Procedural 3D Generation and Parametric Editing of 3D Shapes with Large Language Models
- 用程序化图结构表示3D模型,暴露可调参数
- 自然语言指令下实现秒级修改,效率超传统方法400倍
- 适合需要快速迭代设计的工业建模与创意用户
3D模型生成长期依赖专业技能。现有方法生成非可编辑的网格或点云,难以支持设计迭代。本文提出Proc3D系统,通过引入程序化紧凑图(PCG)表示3D模型,编码生成算法与结构,使关键参数可视化,支持滑块、复选框手动调节,以及基于大语言模型的自然语言实时自动修改。采用GPT-4o与微调后的LLAMA-3两种生成方式。实验表明,Proc3D在编辑效率上超越现有方法400倍以上,且在ULIP评分上提升28%,显著增强文本到3D的一致性。该系统实现了文本对齐的3D生成与精确实时参数化编辑,适用于高精度文本驱动的3D设计场景。
原文摘要 · Abstract (English)
Generating 3D models has traditionally been a complex task requiring specialized expertise. While recent advances in generative AI have sought to automate this process, existing methods produce non-editable representation, such as meshes or point clouds, limiting their adaptability for iterative design. In this paper, we introduce Proc3D, a system designed to generate editable 3D models while enabling real-time modifications. At its core, Proc3D introduces procedural compact graph (PCG), a graph representation of 3D models, that encodes the algorithmic rules and structures necessary for generating the model. This representation exposes key parameters, allowing intuitive manual adjustments via sliders and checkboxes, as well as real-time, automated modifications through natural language prompts using Large Language Models (LLMs). We demonstrate Proc3D's capabilities using two generative approaches: GPT-4o with in-context learning (ICL) and a fine-tuned LLAMA-3 model. Experimental results show that Proc3D outperforms existing methods in editing efficiency, achieving more than 400x speedup over conventional approaches that require full regeneration for each modification. Additionally, Proc3D improves ULIP scores by 28%, a metric that evaluates the alignment between generated 3D models and text prompts. By enabling text-aligned 3D model generation along with precise, real-time parametric edits, Proc3D facilitates highly accurate text-based image editing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。