单图生成物理合理的3D组合资产，通过物理模拟优化位置。
PhyCAGE: Physically Plausible Compositional 3D Asset Generation from a Single Image
- 用多视角图像和3D高斯泼溅表示物体组件。
- 引入物理增强得分蒸馏采样，使物体间位置更符合物理规律。
- 适合需要真实感3D内容的影视与游戏开发场景。
我们提出PhyCAGE，首个从单张图像生成物理合理组合3D资产的方法。给定输入图像后，首先生成资产各组件的一致多视图图像，再使用3D高斯泼溅（3D Gaussian Splatting）进行建模。为确保代表物体的高斯点在空间上物理兼容，我们提出物理模拟增强得分蒸馏采样（PSE-SDS）技术：将SDS损失的梯度作为物理模拟的初始速度，使模拟器作为物理引导优化器，逐步修正高斯点位置至物理相容状态。实验表明，该方法可基于单张图像生成物理合理的组合3D资产。
原文摘要 · Abstract (English)
We present PhyCAGE, the first approach for physically plausible compositional 3D asset generation from a single image. Given an input image, we first generate consistent multi-view images for components of the assets. These images are then fitted with 3D Gaussian Splatting representations. To ensure that the Gaussians representing objects are physically compatible with each other, we introduce a Physical Simulation-Enhanced Score Distillation Sampling (PSE-SDS) technique to further optimize the positions of the Gaussians. It is achieved by setting the gradient of the SDS loss as the initial velocity of the physical simulation, allowing the simulator to act as a physics-guided optimizer that progressively corrects the Gaussians' positions to a physically compatible state. Experimental results demonstrate that the proposed method can generate physically plausible compositional 3D assets given a single image.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。