用外部零件库增强3D生成与编辑,让单图生成更精准、可局部修改。
PartRAG: Retrieval-Augmented Part-Level 3D Generation and Editing
- 引入分层对比检索,从1236个零件资产中匹配图像块与3D零件潜空间。
- 在共享规范空间中实现零件替换、属性调整等局部编辑,5-8秒完成。
- 生成结果更清晰、细长结构更真实,适合需要精细控制的3D设计场景。
单图像3D生成中的部件级结构仍具挑战:学习到的先验难以覆盖部件几何的长尾分布,且难以保持多视角一致性;现有系统对精确、局部编辑支持有限。本文提出PartRAG,一种融合外部零件数据库与扩散变压器的检索增强框架,将生成与可编辑表示结合。为解决第一个问题,提出分层对比检索模块,在部件与物体粒度上对齐密集图像块与3D零件潜空间,从包含1,236个标注零件资产的数据库中检索多样化、物理合理的示例注入去噪过程。为解决第二个问题,引入掩码式部件级编辑器,基于共享规范空间操作,实现部件替换、属性优化和组合更新,无需重新生成整个物体,同时保留非目标部分和多视角一致性。在Objaverse、ShapeNet和ABO上取得竞争力结果:在Objaverse上Chamfer Distance从0.1726降至0.1528,F-Score从0.7472升至0.844;推理耗时38秒,交互编辑仅需5-8秒。定性结果显示,生成物体具有更锐利的部件边界、更好的细长结构保真度,并在可动对象上表现稳健。代码与官网见:https://github.com/AIGeeksGroup/PartRAG, https://aigeeksgroup.github.io/PartRAG。
原文摘要 · Abstract (English)
Single-image 3D generation with part-level structure remains challenging: learned priors struggle to cover the long tail of part geometries and maintain multi-view consistency, and existing systems provide limited support for precise, localized edits. We present PartRAG, a retrieval-augmented framework that integrates an external part database with a diffusion transformer to couple generation with an editable representation. To overcome the first challenge, we introduce a Hierarchical Contrastive Retrieval module that aligns dense image patches with 3D part latents at both part and object granularity, retrieving from a curated bank of 1,236 part-annotated assets to inject diverse, physically plausible exemplars into denoising. To overcome the second challenge, we add a masked, part-level editor that operates in a shared canonical space, enabling swaps, attribute refinements, and compositional updates without regenerating the whole object while preserving non-target parts and multi-view consistency. PartRAG achieves competitive results on Objaverse, ShapeNet, and ABO-reducing Chamfer Distance from 0.1726 to 0.1528 and raising F-Score from 0.7472 to 0.844 on Objaverse-with inference of 38s and interactive edits in 5-8s. Qualitatively, PartRAG produces sharper part boundaries, better thin-structure fidelity, and robust behavior on articulated objects. Code: https://github.com/AIGeeksGroup/PartRAG. Website: https://aigeeksgroup.github.io/PartRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。