无需3D模型或类别先验，实现真实世界下物体姿态与形状的零样本多假设估计。
OmniShape: Zero-Shot Multi-Hypothesis Shape and Pose Estimation in the Real World
- 分离建模观测投影与几何先验，通过条件扩散模型生成多假设解
- 在真实数据集上实现高精度姿态与完整形状估计，支持多模态输出
- 适用于未知物体的实时三维重建，适合机器人、AR等场景
我们希望仅从单张图像中估计物体的姿态和完整形状，且不依赖已知的3D模型或类别信息。本文提出OmniShape，首个实现概率性姿态与形状估计的方法。其核心思想是将形状补全分解为两个多模态分布：一是观测值在由数据集定义的归一化物体参考系中的投影分布；二是以三平面神经场表示的物体几何先验分布。通过分别训练这两个分布的条件扩散模型，可从联合姿态-形状分布中采样多个假设。OmniShape在具有挑战性的真实世界数据集上展现出优异性能。项目网站：https://tri-ml.github.io/omnishape
原文摘要 · Abstract (English)
We would like to estimate the pose and full shape of an object from a single observation, without assuming known 3D model or category. In this work, we propose OmniShape, the first method of its kind to enable probabilistic pose and shape estimation. OmniShape is based on the key insight that shape completion can be decoupled into two multi-modal distributions: one capturing how measurements project into a normalized object reference frame defined by the dataset and the other modelling a prior over object geometries represented as triplanar neural fields. By training separate conditional diffusion models for these two distributions, we enable sampling multiple hypotheses from the joint pose and shape distribution. OmniShape demonstrates compelling performance on challenging real world datasets. Project website: https://tri-ml.github.io/omnishape
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。