用多相机系统实现野外真实场景下高精度手部三维追踪
Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rig
- 结合背挂八相机与头戴设备，融合远距与第一视角数据
- 构建同步多视图数据集，手部三维标注误差显著降低
- 适合研究真实场景中手物交互的视觉算法开发者
在非受限环境下精确追踪手部及其与世界的交互，仍是第一人称计算机视觉的重要挑战。现有数据集大多来自受控实验室环境，限制了场景多样性与模型泛化能力。为此，我们提出一种无标记多相机系统，可在真正野外条件下捕捉精确的3D手部与物体信息。系统包含一个轻量级背挂式八相机阵列和用户佩戴的Meta Quest 3头显（提供两个第一视角）。我们设计了跨视角追踪流程，从该系统生成高精度3D手部姿态真值，并严格评估其质量。通过采集包含同步多视图图像与精确3D手部姿态的标注数据集，证明该方法能显著缓解环境真实度与3D标注精度之间的权衡。
原文摘要 · Abstract (English)
Accurate 3D tracking of hands and their interactions with the world in unconstrained settings remains a significant challenge for egocentric computer vision. With few exceptions, existing datasets are predominantly captured in controlled lab setups, limiting environmental diversity and model generalization. To address this, we introduce a novel marker-less multi-camera system designed to capture precise 3D hands and objects, which allows for nearly unconstrained mobility in genuinely in-the-wild conditions. We combine a lightweight, back-mounted capture rig with eight exocentric cameras, and a user-worn Meta Quest 3 headset, which contributes two egocentric views. We design an ego-exo tracking pipeline to generate accurate 3D hand pose ground truth from this system, and rigorously evaluate its quality. By collecting an annotated dataset featuring synchronized multi-view images and precise 3D hand poses, we demonstrate the capability of our approach to significantly reduce the trade-off between environmental realism and 3D annotation accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。