跨动物类别统一提升2D关节点为3D，兼顾时序一致性与泛化能力。
Object Agnostic 3D Lifting in Space and Time
- 利用相似物体的通用信息增强小样本下的3D重建效果
- 通过时间邻近上下文窗口提升序列内动作的一致性
- 适用于多种动物类别，尤其适合数据稀缺场景
本文提出一种时空联合的、类别无关的2D关键点3D重建方法。现有方法要么仅处理单帧且不依赖物体类别，要么能建模时空依赖但仅限单一物体类别。本方法基于两大原则：一是利用相似物体的通用信息，在物体特异性训练数据少时提升性能；二是利用时间邻近的上下文窗口，实现序列内的一致性。实验表明，该方法在多种动物类别上，于帧级和序列级指标均优于当前最优方法。最后，我们发布了包含多种动物类别的新合成数据集，含3D骨架与运动序列。
原文摘要 · Abstract (English)
We present a spatio-temporal perspective on category-agnostic 3D lifting of 2D keypoints over a temporal sequence. Our approach differs from existing state-of-the-art methods that are either: (i) object-agnostic, but can only operate on individual frames, or (ii) can model space-time dependencies, but are only designed to work with a single object category. Our approach is grounded in two core principles. First, general information about similar objects can be leveraged to achieve better performance when there is little object-specific training data. Second, a temporally-proximate context window is advantageous for achieving consistency throughout a sequence. These two principles allow us to outperform current state-of-the-art methods on per-frame and per-sequence metrics for a variety of animal categories. Lastly, we release a new synthetic dataset containing 3D skeletons and motion sequences for a variety of animal categories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。