用代码生成精准工业场景，让大模型搞定高精度布局。
SceneGenAgent: Precise Industrial Scene Generation with Coding Agent
- 通过C#代码结构化生成，实现空间布局精确规划。
- 实测成功率达81.0%，满足工业场景定量需求。
- 适合需要高精度3D场景生成的制造仿真研究者。
工业场景建模对制造仿真至关重要。尽管大语言模型（LLMs）在从文本生成通用3D场景方面取得显著进展，但生成需精确尺寸与定位的工业场景仍具挑战性，要求复杂的空间布局规划。为此，我们提出SceneGenAgent，一个基于LLM的代理，通过编写C#代码生成工业场景。该方法通过结构化可计算格式、布局验证与迭代优化，确保布局符合工业场景的定量要求。实验表明，搭载SceneGenAgent的LLMs在真实工业场景生成任务中成功率最高达81.0%，有效满足多数生成需求。为提升可及性，我们构建了SceneInstruct数据集，用于微调开源LLM以接入SceneGenAgent。实验显示，在SceneInstruct上微调的Llama3.1-70B性能显著提升，接近GPT-4o水平。代码与数据已公开于https://github.com/THUDM/SceneGenAgent。
原文摘要 · Abstract (English)
The modeling of industrial scenes is essential for simulations in industrial manufacturing. While large language models (LLMs) have shown significant progress in generating general 3D scenes from textual descriptions, generating industrial scenes with LLMs poses a unique challenge due to their demand for precise measurements and positioning, requiring complex planning over spatial arrangement. To address this challenge, we introduce SceneGenAgent, an LLM-based agent for generating industrial scenes through C# code. SceneGenAgent ensures precise layout planning through a structured and calculable format, layout verification, and iterative refinement to meet the quantitative requirements of industrial scenarios. Experiment results demonstrate that LLMs powered by SceneGenAgent exceed their original performance, reaching up to 81.0% success rate in real-world industrial scene generation tasks and effectively meeting most scene generation requirements. To further enhance accessibility, we construct SceneInstruct, a dataset designed for fine-tuning open-source LLMs to integrate into SceneGenAgent. Experiments show that fine-tuning open-source LLMs on SceneInstruct yields significant performance improvements, with Llama3.1-70B approaching the capabilities of GPT-4o. Our code and data are available at https://github.com/THUDM/SceneGenAgent .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。