从野外视频中自监督学习统一物体坐标系，无需人工标注。
Emergence of a Shared Canonical Object Frame from In-the-Wild Videos
- 通过共享粗略网格构建几何瓶颈，实现跨视频对齐。
- 在16万段野外视频上训练，达到与有标注方法相当的精度。
- 适合做无监督姿态估计、物体识别与3D重建的研究者。
跨不同实例比较物体朝向和位置需将其姿态表达在统一的规范坐标系中。传统方法依赖人工标注建立该坐标系，存在扩展瓶颈，限制了类别和实例多样性。本文表明，仅使用来自运动恢复结构（SfM）的噪声相机位姿，即可在真实场景拍摄的以物体为中心的视频上通过自监督训练，让统一的规范坐标系自发产生。核心思想是将所有训练序列经由一个不包含类别特异性细节的粗粒度规范网格（coarse canonical mesh），通过学习像素到网格的密集对应关系，并利用噪声SfM几何信息估计每序列的对齐方式，从而基于多视角一致性及特征提取器的语义先验，自然形成共享规范坐标系，无需任何规范姿态标签或类别条件。在16万段真实场景物体视频上自监督训练后，本方法在类别级姿态估计基准测试中表现媲美依赖规范姿态监督的方法。代码与模型权重已开源：https://github.com/Fischer-Tom/Emergent-Canonical-Frame/。
原文摘要 · Abstract (English)
Comparing object orientations and positions across different instances requires their poses to be expressed in a shared canonical frame. Establishing such frames has traditionally required manual annotation, creating a scaling bottleneck that limits category and instance diversity. We show that a shared canonical frame can instead emerge from self-supervised training on object-centric videos captured in the wild, using only noisy camera poses from Structure-from-Motion. Our key idea is to route all training sequences through a shared geometric bottleneck: a coarse canonical mesh that carries no category-specific detail. By learning dense correspondences from image pixels to this mesh, and estimating per-sequence alignments from noisy SfM geometry, a common canonical frame emerges from multi-view consistency and the semantic priors of the feature extractor, without any canonical pose labels or category conditioning. Trained in a self-supervised manner on 160,000 in-the-wild object videos, our method achieves competitive accuracy on category-level pose estimation benchmarks compared to methods that rely on canonical pose supervision. The code and checkpoint is available on https://github.com/Fischer-Tom/Emergent-Canonical-Frame/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。