用照片视频推断物体内部结构，速度更快、精度更高。
TopoGaussian: Inferring Internal Topology Structures from Visual Clues
- 基于粒子的可微仿真器，融合材质、执行器和碰撞机制。
- 比传统网格方法快5.26倍，且形状质量更优。
- 适合3D视觉、软体机器人与制造领域应用。
我们提出TopoGaussian，一种基于粒子的端到端方法，仅需易获取的照片和视频即可推断不透明物体的内部拓扑结构。传统网格方法需繁琐的网格填充与修复，且输出粗糙边界表面。本方法结合高斯点阵与新型通用粒子可微仿真器，可同时建模材料属性、执行器和碰撞，且不依赖网格。基于该仿真器的梯度，优化时可灵活选择粒子、神经隐式曲面或二次曲面等拓扑表示。在合成数据集及四个真实世界任务（含3D打印原型）中验证了有效性。相比现有网格方法，平均提速5.26倍，且形状质量更优。结果表明该方法在3D视觉、软体机器人与制造领域具有潜力。
原文摘要 · Abstract (English)
We present TopoGaussian, a holistic, particle-based pipeline for inferring the interior structure of an opaque object from easily accessible photos and videos as input. Traditional mesh-based approaches require tedious and error-prone mesh filling and fixing process, while typically output rough boundary surface. Our pipeline combines Gaussian Splatting with a novel, versatile particle-based differentiable simulator that simultaneously accommodates constitutive model, actuator, and collision, without interference with mesh. Based on the gradients from this simulator, we provide flexible choice of topology representation for optimization, including particle, neural implicit surface, and quadratic surface. The resultant pipeline takes easily accessible photos and videos as input and outputs the topology that matches the physical characteristics of the input. We demonstrate the efficacy of our pipeline on a synthetic dataset and four real-world tasks with 3D-printed prototypes. Compared with existing mesh-based method, our pipeline is 5.26x faster on average with improved shape quality. These results highlight the potential of our pipeline in 3D vision, soft robotics, and manufacturing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。