arXiv:2505.07747cs.CV2025-05被引 93

开源3D资产生成框架,支持高保真可控建模与2D控制技术迁移。

Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets

  • 基于500万+资产清洗,构建200万高质量标准化3D数据集。
  • 两阶段架构:混合VAE-DiT生成几何,扩散模型保证纹理跨视角一致。
  • 首次实现2D控制方法(如LoRA)直接用于3D生成,适合工业设计与游戏开发。

尽管生成式人工智能在文本、图像、音频和视频领域取得显著进展,但3D生成仍因数据稀缺、算法限制和生态碎片化而相对滞后。为此,我们提出Step1X-3D,一个开放框架,通过:(1) 严格的数据清洗流程处理超500万资产,构建200万高质量数据集,具备标准化几何与纹理属性;(2) 两阶段3D原生架构,结合混合VAE-DiT几何生成器与基于扩散的纹理合成模块;(3) 全开源发布模型、训练代码与适配模块。几何生成部分采用基于Perceiver的潜在编码与锐利边缘采样以保留细节,纹理合成模块则通过几何条件与潜空间同步确保跨视角一致性。基准测试显示其性能超越现有开源方法,并媲美专有解决方案。尤为关键的是,该框架首次实现2D控制技术(如LoRA)直接迁移至3D生成,打通2D与3D生成范式。通过同时提升数据质量、算法保真度与可复现性,Step1X-3D旨在为可控3D资产生成建立新的开源研究标准。

原文摘要 · Abstract (English)

While generative artificial intelligence has advanced significantly across text, image, audio, and video domains, 3D generation remains comparatively underdeveloped due to fundamental challenges such as data scarcity, algorithmic limitations, and ecosystem fragmentation. To this end, we present Step1X-3D, an open framework addressing these challenges through: (1) a rigorous data curation pipeline processing >5M assets to create a 2M high-quality dataset with standardized geometric and textural properties; (2) a two-stage 3D-native architecture combining a hybrid VAE-DiT geometry generator with an diffusion-based texture synthesis module; and (3) the full open-source release of models, training code, and adaptation modules. For geometry generation, the hybrid VAE-DiT component produces TSDF representations by employing perceiver-based latent encoding with sharp edge sampling for detail preservation. The diffusion-based texture synthesis module then ensures cross-view consistency through geometric conditioning and latent-space synchronization. Benchmark results demonstrate state-of-the-art performance that exceeds existing open-source methods, while also achieving competitive quality with proprietary solutions. Notably, the framework uniquely bridges the 2D and 3D generation paradigms by supporting direct transfer of 2D control techniques~(e.g., LoRA) to 3D synthesis. By simultaneously advancing data quality, algorithmic fidelity, and reproducibility, Step1X-3D aims to establish new standards for open research in controllable 3D asset generation.

3D生成扩散模型开源框架纹理合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。