用大模型自动编程生成超1万件关节3D资产，解决数据短缺难题
Articraft: An Agentic System for Scalable Articulated 3D Asset Generation
- 让大模型写程序构建3D关节物体，通过专用开发接口实现自动化生成
- 产出超过10,000件跨245类的高质量关节资产，质量优于现有工具
- 适合机器人仿真、虚拟现实等需要丰富3D物理对象的场景
理解关节3D物体的一大瓶颈是缺乏大规模多样数据集。本文提出利用大语言模型（LLM）弥补这一缺口，实现关节资产的规模化生成。我们将生成关节3D资产的问题转化为编写构建它的程序。为此引入新代理系统Articraft，能自动编写此类程序。设计了面向特定领域的编程接口，并通过配套工具链帮助LLM高效工作。该系统支持定义部件、组合几何体、指定关节并编写验证测试。工具链为LLM提供受限工作空间与接口，验证生成资产并返回结构化反馈，避免其陷入如编写URDF文件或管理复杂环境等细节。实验表明，该方法生成的资产质量高于当前最先进的关节资产生成器及通用编码代理。基于此，我们构建了包含超过10,000件资产的Articraft-10K数据集，覆盖245个类别，验证其在训练关节资产模型及机器人仿真、虚拟现实等下游任务中的实用性。
原文摘要 · Abstract (English)
A bottleneck in learning to understand articulated 3D objects is the lack of large and diverse datasets. In this paper, we propose to leverage large language models (LLMs) to close this gap and generate articulated assets at scale. We reduce the problem of generating an articulated 3D asset to that of writing a program that builds it. We then introduce a new agentic system, Articraft, that writes such programs automatically. We design a programmatic interface and harness to help the LLM do so effectively. The LLM writes code against a domain-specific SDK for defining parts, composing geometry, specifying joints, and writing tests to validate the resulting assets. The harness exposes a restricted workspace and interface to the LLM, validates the resulting assets, and returns structured feedback. In this way, the LLM is not distracted by details such as authoring a URDF file or managing a complex software environment. We show that this produces higher-quality assets than both state-of-the-art articulated-asset generators and general-purpose coding agents. Using Articraft, we build Articraft-10K, a curated dataset of over 10K articulated assets spanning 245 categories, and show its utility both for training models of articulated assets and in downstream applications such as robotics simulation and virtual reality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。