arXiv:2608.23869cs.CV2026-08

将生成的3D网格自动分解为物理可模拟的材质组件。

Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decomposition

论文配图:Gen2Physics: Grounding Generated 3D Meshes in Physics via Multi-View Material Decomposition
图 1 · 摘自论文原文
  • 用视觉变压器实现密集材质分割,直接处理网格而非体素。
  • 材料分割准确率从15.6提升至48.3 mIoU,且输出封闭子网格。
  • 结合语言模型推理材质属性与内部结构,适合游戏与机器人仿真。

当前顶尖生成模型虽能产出高保真3D网格,但缺乏交互模拟、游戏或机器人所需的物理属性。本文提出Gen2Physics,一种统一且自动化的框架,通过将生成网格分解为材质成分来赋予其物理特性。不同于依赖体素表示且不兼容标准物理引擎的方法,Gen2Physics直接在网格上操作,生成即刻可用的仿真资产。该流程整合了微调的Vision Transformer进行密集材质分割、鲁棒的2D到3D一致性投影,以及由视觉-语言模型(VLM)引导的精炼模块,利用上下文推理分配物理属性并推断内部几何结构(实心或中空)。通过将表面区域转为具有不同密度的体积,方法支持物理上合理的动态模拟。在ABO-500与PartNet-Material数据集上的实验表明,该方法使材料分割准确率从15.6提升至48.3 mIoU,超过两倍;质量估计精度与体素方法相当,并是唯一能输出防水封闭材质子网格的方法。

原文摘要 · Abstract (English)

While state-of-the-art generative models produce high-fidelity 3D meshes, these outputs lack the physical properties required for interactive simulation, gaming, or robotics. We introduce Gen2Physics, a unified and automated framework that grounds generated meshes in physics by automatically decomposing them into their constituent material components. Unlike prior approaches, which focus on volumetric representations incompatible with standard physics engines, Gen2Physics operates directly on meshes to produce immediately simulation-ready assets. Our pipeline integrates a fine-tuned Vision Transformer for dense material segmentation, a robust 2D-to-3D consistency projection, and a Vision-Language Model (VLM) guided refinement that leverages contextual reasoning to assign physical properties and infer internal geometry (solid vs. hollow). By converting surface patches into volumes with distinct densities, our method enables physically plausible dynamic simulations. Experimental results on the ABO-500 and PartNet-Material benchmarks demonstrate that Gen2Physics more than doubles the material segmentation accuracy of prior physics-grounding pipelines (15.6 to 48.3 mIoU), while matching the mass-estimation accuracy of volumetric methods and being the only approach to output watertight per-material sub-meshes.

3D生成物理模拟材质分割网格处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。