arXiv:2411.16800cs.CV2024-11中稿 · ACM MM 2025被引 20

自动感知多材质结构,生成物理真实的4D动态内容

Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception

  • 通过语义分割自动划分3D表面的材质区域
  • 可构建物体内部机械结构并自动识别材质属性
  • 无需人工设定物理参数,适合复杂多材料场景

4D内容生成旨在创建对输入对象(如图像或3D表示)有响应的动态演化3D内容。现有方法虽引入物理先验来驱动3D动画,但存在两大瓶颈:一是用户需手动指定材料属性,缺乏物理知识者难以操作;二是难以有效处理多材质复合物体的生成。为此,我们提出Phys4DGen,一种融合多材质组合感知与物理模拟的新型4D生成框架。该框架包含三个创新模块:首先,3D材质分组模块通过语义分割自动划分3D表面异质材质区域;其次,内部物理结构发现模块构建物体内部力学结构;最后,从多模态大语言模型中提炼物理先验知识,实现对物体表面和内部材质属性的快速、自动识别。在合成与真实世界数据集上的实验表明,Phys4DGen可在开放世界场景下生成高保真、物理合理的4D内容,显著优于当前最优方法。

原文摘要 · Abstract (English)

4D content generation aims to create dynamically evolving 3D content that responds to specific input objects such as images or 3D representations. Current approaches typically incorporate physical priors to animate 3D representations, but these methods suffer from significant limitations: they not only require users lacking physics expertise to manually specify material properties but also struggle to effectively handle the generation of multi-material composite objects. To address these challenges, we propose Phys4DGen, a novel 4D generation framework that integrates multi-material composition perception with physical simulation. The framework achieves automated, physically plausible 4D generation through three innovative modules: first, the 3D Material Grouping module partitions heterogeneous material regions on 3D representations' surfaces via semantic segmentation; second, the Internal Physical Structure Discovery module constructs the mechanical structure of object interiors; finally, we distill physical prior knowledge from multimodal large language models to enable rapid and automatic material properties identification for both objects' surfaces and interiors. Experiments on both synthetic and real-world datasets demonstrate that Phys4DGen can generate high-fidelity 4D content with physical realism in open-world scenarios, significantly outperforming state-of-the-art methods.

4D生成物理模拟多材质自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。