用Transformer从零散图像重建可动3D物体，效果超越现有方法。
ART: Articulated Reconstruction Transformer
- 将可动物体拆解为刚性部件，通过部分槽位联合预测几何与关节参数。
- 在多状态图像上实现高精度重建，物理可解释且支持仿真导出。
- 无需类别限制，适用于多种可动物体，适合机器人与虚拟现实应用。
我们提出ART（Articulated Reconstruction Transformer），一种无类别依赖的前馈模型，仅需稀疏的多状态RGB图像即可重建完整的3D可动物体。以往方法或依赖缓慢的优化和脆弱的状态间对应关系，或局限于特定物体类别。ART将可动物体视为刚性部件的组合，将重建问题建模为基于部件的预测。其创新的Transformer架构将稀疏图像输入映射到一组可学习的部件槽位，从中联合解码每个部件的统一表示，包括3D几何、纹理及显式关节参数。重建结果具备物理可解释性，可直接用于仿真。模型在大规模、多样化的数据集上以逐部件监督进行训练，并在多个基准上评估，显著优于现有基线，建立了基于图像的可动物体重建新基准。
原文摘要 · Abstract (English)
We introduce ART, Articulated Reconstruction Transformer -- a category-agnostic, feed-forward model that reconstructs complete 3D articulated objects from only sparse, multi-state RGB images. Previous methods for articulated object reconstruction either rely on slow optimization with fragile cross-state correspondences or use feed-forward models limited to specific object categories. In contrast, ART treats articulated objects as assemblies of rigid parts, formulating reconstruction as part-based prediction. Our newly designed transformer architecture maps sparse image inputs to a set of learnable part slots, from which ART jointly decodes unified representations for individual parts, including their 3D geometry, texture, and explicit articulation parameters. The resulting reconstructions are physically interpretable and readily exportable for simulation. Trained on a large-scale, diverse dataset with per-part supervision, and evaluated across diverse benchmarks, ART achieves significant improvements over existing baselines and establishes a new state of the art for articulated object reconstruction from image inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。