用大模型写代码生成3D资产,可编辑可协作,比直接生成更灵活。
LL3M: Large Language 3D Modelers
- 将3D生成转为写Python代码,利用Blender API实现精准控制
- 支持风格材质修改、用户迭代优化,生成的代码可读可调试
- 多智能体协同+文档检索增强,让代码自检与人类协作更高效
我们提出LL3M,一个基于预训练大语言模型的多智能体系统,通过在Blender中编写可解释的Python代码生成3D资产。不同于传统从3D数据学习的生成方式,我们把形状生成重新定义为代码写作任务,提升模块化、可编辑性,并更好融入艺术家工作流程。给定文本提示后,LL3M协调多个专用语言模型智能体完成规划、检索、代码编写、调试与优化,生成可操作的Blender脚本,实现几何与外观的构建与修改。生成的代码作为场景与对象的高层、可读、可文档化的表示,充分使用B-mesh、几何修饰符、着色节点等复杂功能,支持多样、无约束的形状、材质与场景生成。该代码形式便于后续智能体或人类通过代码调整或程序参数进行迭代优化,自然形成人机共创循环:智能体可基于代码与视觉自我评估,用户指令则提供直观的精细化路径。各智能体共享代码上下文,具备对先前尝试的认知;结合Blender API文档构建的检索增强生成知识库(BlenderRAG),赋予智能体操作示例、类型与函数信息,支撑高级建模与代码正确性。我们在多种形状类别、风格材质编辑及用户驱动优化任务中验证了LL3M的有效性。实验表明,代码作为生成与可解释介质,在3D资产创作中具有强大潜力。项目页面见https://threedle.github.io/ll3m。
原文摘要 · Abstract (English)
We present LL3M, a multi-agent system that leverages pretrained large language models (LLMs) to generate 3D assets by writing interpretable Python code in Blender. We break away from the typical generative approach that learns from a collection of 3D data. Instead, we reformulate shape generation as a code-writing task, enabling greater modularity, editability, and integration with artist workflows. Given a text prompt, LL3M coordinates a team of specialized LLM agents to plan, retrieve, write, debug, and refine Blender scripts that generate and edit geometry and appearance. The generated code works as a high-level, interpretable, human-readable, well-documented representation of scenes and objects, making full use of sophisticated Blender constructs (e.g. B-meshes, geometry modifiers, shader nodes) for diverse, unconstrained shapes, materials, and scenes. This code presents many avenues for further agent and human editing and experimentation via code tweaks or procedural parameters. This medium naturally enables a co-creative loop in our system: agents can automatically self-critique using code and visuals, while iterative user instructions provide an intuitive way to refine assets. A shared code context across agents enables awareness of previous attempts, and a retrieval-augmented generation knowledge base built from Blender API documentation, BlenderRAG, equips agents with examples, types, and functions empowering advanced modeling operations and code correctness. We demonstrate the effectiveness of LL3M across diverse shape categories, style and material edits, and user-driven refinements. Our experiments showcase the power of code as a generative and interpretable medium for 3D asset creation. Our project page is at https://threedle.github.io/ll3m.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。