arXiv:2506.11148cs.CVcs.LG2025-06被引 5

让大模型生成的3D物体自动符合物理规律,提升工程设计可行性。

LLM-to-Phy3D: Physically Conform Online 3D Object Generation with LLMs

  • 通过视觉与物理双重反馈,实时优化大模型生成的3D对象。
  • 在车辆设计中,物理合规性提升4.5%至106.7%。
  • 适合需要真实物理可行性的工业设计与AI生成应用。

生成式人工智能和大语言模型(LLMs)已革新多模态数字内容创作,但在工程设计所需的物理可行物体制作方面仍鲜有探索。现有LLM-to-3D模型缺乏物理知识,导致输出脱离现实约束。为此,我们提出LLM-to-Phy3D,一种在线物理合规3D对象生成方法,使现有LLM-to-3D模型可实时生成符合物理规律的3D对象。该方法引入新型在线黑箱优化循环,通过视觉与物理评估协同反馈,在迭代优化中引导大模型发现能生成更高物理性能和几何新颖性的提示。系统评估显示,在车辆设计优化任务中,相比传统模型,其物理合规目标3D设计生成能力提升4.5%至106.7%。结果表明,该方法具备在物理智能领域广泛应用于科学与工程设计的潜力。

原文摘要 · Abstract (English)

The emergence of generative artificial intelligence (GenAI) and large language models (LLMs) has revolutionized the landscape of digital content creation in different modalities. However, its potential use in Physical AI for engineering design, where the production of physically viable artifacts is paramount, remains vastly underexplored. The absence of physical knowledge in existing LLM-to-3D models often results in outputs detached from real-world physical constraints. To address this gap, we introduce LLM-to-Phy3D, a physically conform online 3D object generation that enables existing LLM-to-3D models to produce physically conforming 3D objects on the fly. LLM-to-Phy3D introduces a novel online black-box refinement loop that empowers large language models (LLMs) through synergistic visual and physics-based evaluations. By delivering directional feedback in an iterative refinement process, LLM-to-Phy3D actively drives the discovery of prompts that yield 3D artifacts with enhanced physical performance and greater geometric novelty relative to reference objects, marking a substantial contribution to AI-driven generative design. Systematic evaluations of LLM-to-Phy3D, supported by ablation studies in vehicle design optimization, reveal various LLM improvements gained by 4.5% to 106.7% in producing physically conform target domain 3D designs over conventional LLM-to-3D models. The encouraging results suggest the potential general use of LLM-to-Phy3D in Physical AI for scientific and engineering applications.

3D生成物理模拟大模型设计优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。