用检索增强生成技术,让3D网格自动生成更快更准且支持增量编辑。
Mesh RAG: Retrieval Augmentation for Autoregressive Mesh Generation
- 通过点云分割与配准检索组件,打破传统序列生成依赖。
- 在多个模型上实现质量提升、速度加快,且无需重新训练。
- 适合需要快速迭代设计的工业建模与游戏开发场景。
3D网格是工业设计、游戏开发、仿真和机器人等领域的重要基础。传统上由艺术家手动创建,耗时且难以扩展。为自动化资产生成,自回归模型已成为主流方法。然而,现有提升质量的方式多依赖更大模型或更长序列,导致生成时间变长,且固有的顺序性造成严重质量-速度权衡,也极大限制了增量编辑。为此,我们提出 Mesh RAG,一种无需训练、即插即用的新型框架,用于增强自回归网格生成模型。受语言模型 RAG 启发,该方法利用点云分割、空间变换和点云配准,实现组件的检索、生成与融合。此检索式策略解耦了生成过程对严格顺序的依赖,支持高效并行推理。我们在多个基础自回归网格生成模型上验证了 Mesh RAG 的广泛适用性,结果表明其显著提升了网格质量,加速了生成速度,相比序列化部件预测更优,并支持增量编辑,所有改进均无需模型重训练。
原文摘要 · Abstract (English)
3D meshes are a critical building block for applications ranging from industrial design and gaming to simulation and robotics. Traditionally, meshes are crafted manually by artists, a process that is time-intensive and difficult to scale. To automate and accelerate this asset creation, autoregressive models have emerged as a powerful paradigm for artistic mesh generation. However, current methods to enhance quality typically rely on larger models or longer sequences that result in longer generation time, and their inherent sequential nature imposes a severe quality-speed trade-off. This sequential dependency also significantly complicates incremental editing. To overcome these limitations, we propose Mesh RAG, a novel, training-free, plug-and-play framework for autoregressive mesh generation models. Inspired by RAG for language models, our approach augments the generation process by leveraging point cloud segmentation, spatial transformation, and point cloud registration to retrieve, generate, and integrate mesh components. This retrieval-based approach decouples generation from its strict sequential dependency, facilitating efficient and parallelizable inference. We demonstrate the wide applicability of Mesh RAG across various foundational autoregressive mesh generation models, showing it significantly enhances mesh quality, accelerates generation speed compared to sequential part prediction, and enables incremental editing, all without model retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。