用大模型一键生成编辑3D服装,支持图文输入与对话交互。
ChatGarment: Garment Estimation, Generation and Editing via Large Language Models

- 通过微调视觉语言模型直接输出含风格与参数的JSON文件
- 可从真实照片或文本生成服装剪裁图,并在3D人体上动态展示
- 适合时尚设计、游戏道具快速开发,无需专业建模技能
我们提出ChatGarment,一种利用大视觉语言模型(VLM)自动化实现3D服装的估计、生成与编辑的新方法。相比以往方法在真实场景下表现差或缺乏交互编辑能力,ChatGarment可从真实图像或草图中估计缝制图,也可根据文本描述生成服装,并基于用户指令进行交互式修改,所有操作均在对话中完成。生成的缝制图可进一步披挂在3D人体上并动画化。该系统通过微调VLM,直接输出包含服装类型、风格描述及连续数值属性的JSON文件,再由参数化编程模型生成实际缝制图。为此,我们改进了现有模型GarmentCode,扩大了服装类型覆盖范围并简化结构以利于高效微调。同时,构建了一个大规模图像-缝制图与文本-缝制图配对数据集,通过自动化数据管道生成。大量评估表明,ChatGarment能准确重建、生成与编辑多模态输入下的服装,展现出在时尚与游戏领域的巨大应用潜力。代码与数据见https://chatgarment.github.io/。
原文摘要 · Abstract (English)
We introduce ChatGarment, a novel approach that leverages large vision-language models (VLMs) to automate the estimation, generation, and editing of 3D garments from images or text descriptions. Unlike previous methods that struggle in real-world scenarios or lack interactive editing capabilities, ChatGarment can estimate sewing patterns from in-the-wild images or sketches, generate them from text descriptions, and edit garments based on user instructions, all within an interactive dialogue. These sewing patterns can then be draped on a 3D body and animated. This is achieved by finetuning a VLM to directly generate a JSON file that includes both textual descriptions of garment types and styles, as well as continuous numerical attributes. This JSON file is then used to create sewing patterns through a programming parametric model. To support this, we refine the existing programming model, GarmentCode, by expanding its garment type coverage and simplifying its structure for efficient VLM fine-tuning. Additionally, we construct a large-scale dataset of image-to-sewing-pattern and text-to-sewing-pattern pairs through an automated data pipeline. Extensive evaluations demonstrate ChatGarment's ability to accurately reconstruct, generate, and edit garments from multimodal inputs, highlighting its potential to simplify workflows in fashion and gaming applications. Code and data are available at https://chatgarment.github.io/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。