arXiv:2504.10316cs.CV2025-04被引 1

用简单文本生成高质量3D内容,还能自由控制风格和细节。

ESCT3D: Efficient and Selectively Controllable Text-Driven 3D Content Generation with Gaussian Splatting

  • 用GPT-4V自动优化提示词,一次生成满意结果。
  • 支持风格、边缘、草图、姿态等多条件联合控制。
  • 融合多视角深度、掩码等信息,解决3D生成歧义问题。

近年来,文本驱动的3D内容生成取得了显著进展,但仍面临挑战。实际应用中,用户常提供极简文本却期望高质量3D结果,而现有模型对输入提示质量依赖性强,生成过程高度随机,往往需多次迭代才能满足需求,效率低下。为此,我们引入GPT-4V进行自优化,显著提升单次生成满意度。同时,拓展了可控性:用户不仅可输入文本,还可指定风格、边缘、草图、姿态或组合条件,实现更精准控制。训练阶段,我们有效整合多视图深度、掩码、特征与图像信息,缓解3D生成中的常见歧义问题(Janus问题)。大量实验表明,该方法具备强泛化能力,能高效生成高质量且可控的3D内容。

原文摘要 · Abstract (English)

In recent years, significant advancements have been made in text-driven 3D content generation. However, several challenges remain. In practical applications, users often provide extremely simple text inputs while expecting high-quality 3D content. Generating optimal results from such minimal text is a difficult task due to the strong dependency of text-to-3D models on the quality of input prompts. Moreover, the generation process exhibits high variability, making it difficult to control. Consequently, multiple iterations are typically required to produce content that meets user expectations, reducing generation efficiency. To address this issue, we propose GPT-4V for self-optimization, which significantly enhances the efficiency of generating satisfactory content in a single attempt. Furthermore, the controllability of text-to-3D generation methods has not been fully explored. Our approach enables users to not only provide textual descriptions but also specify additional conditions, such as style, edges, scribbles, poses, or combinations of multiple conditions, allowing for more precise control over the generated 3D content. Additionally, during training, we effectively integrate multi-view information, including multi-view depth, masks, features, and images, to address the common Janus problem in 3D content generation. Extensive experiments demonstrate that our method achieves robust generalization, facilitating the efficient and controllable generation of high-quality 3D content.

3D生成文本控制高斯点云可控生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。