用骨骼控制3D生成,让模型按指定结构造物。
SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation
- 将3D骨骼作为核心控制信号,通过轻量适配器注入生成模型。
- 在24000对文本-网格-骨骼数据上训练,实现精准结构控制。
- 支持局部编辑,可基于骨骼调整已有3D资产的特定区域。
原生3D生成模型虽在保真度和速度上表现优异,却难以精确控制结构姿态。本文提出SK-Adapter,一种轻量高效的结构控制框架,首次将3D骨骼作为第一类控制信号。该方法通过可学习的令牌编码关节坐标与拓扑结构,并经交叉注意力注入冻结的3D生成主干网络,使模型能有效关注特定结构约束,同时保留原始生成先验。为弥补数据不足,我们构建了包含24,000个文本-网格-骨骼对的Objaverse-TMS数据集。大量实验表明,该方法在保持几何与纹理质量的前提下,显著优于现有基线,实现了鲁棒的结构控制。进一步拓展至局部3D编辑,支持基于骨骼引导的区域精细化修改,这是以往方法无法实现的。
原文摘要 · Abstract (English)
Native 3D generative models have achieved remarkable fidelity and speed, yet they suffer from a critical limitation: inability to prescribe precise structural articulations, where precise structural control within the native 3D space remains underexplored. This paper proposes SK-Adapter, a simple yet efficient and effective framework that unlocks precise skeletal manipulation for native 3D generation. Moving beyond text or image prompts, which can be ambiguous for precise structure, we treat the 3D skeleton as a first-class control signal. SK-Adapter is a lightweight structural adapter network that encodes joint coordinates and topology into learnable tokens, which are injected into the frozen 3D generation backbone via cross-attention. This design allows the model to not only effectively ``attend'' to specific 3D structural constraints but also preserve its original generative priors. To bridge the data gap, we contribute the Objaverse-TMS dataset, a large-scale dataset of 24k text-mesh-skeleton pairs. Extensive experiments confirm that our method achieves robust structural control while preserving the geometry and texture quality of the foundation model, significantly outperforming existing baselines. Furthermore, we extend this capability to local 3D editing, enabling region-specific editing of existing assets with skeletal guidance, which is unattainable by previous methods. Project page: https://sk-adapter.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。