无需预训练模型，用光场图像实现复杂物体6自由度精准跟踪
Light Field Based 6DoF Tracking of Previously Unobserved Objects
- 基于光场图像提取语义与几何特征，生成视点相关的高斯斑点作为统一表征
- 在含强反光等复杂外观的物体上，性能媲美顶尖模型依赖型追踪器
- 适合需要泛化到未知物体的机器人、自动驾驶系统使用
物体跟踪是机器人和自动驾驶中的关键步骤，需能泛化至此前未见的复杂物体。现有高性能方法通常依赖预先捕获的物体视角构建显式参考模型，限制于已知物体集合。但此类模型在视觉复杂的外观（如反射）下表现不佳。本文提出一种基于光场图像的6自由度物体跟踪方法，无需预训练模型，且对复杂视觉行为（如反射）具有鲁棒性。通过视觉基础模型从光场输入中提取语义与几何特征，并转换为视点依赖的高斯斑点，形成统一表示，支持可微渲染与位姿优化。我们还构建了一个包含挑战性反光物体的光场物体跟踪数据集，提供精确真实位姿。实验表明，该方法在这些困难场景下性能可媲美最先进模型依赖型追踪器，为机器人系统实现通用物体跟踪铺平道路。代码与数据见：https://github.com/nagonch/LiFT-6DoF。
原文摘要 · Abstract (English)
Object tracking is an important step in robotics and reautonomous driving pipelines, which has to generalize to previously unseen and complex objects. Existing high-performing methods often rely on pre-captured object views to build explicit reference models, which restricts them to a fixed set of known objects. However, such reference models can struggle with visually complex appearance, reducing the quality of tracking. In this work, we introduce an object tracking method based on light field images that does not depend on a pre-trained model, while being robust to complex visual behavior, such as reflections. We extract semantic and geometric features from light field inputs using vision foundation models and convert them into view-dependent Gaussian splats. These splats serve as a unified object representation, supporting differentiable rendering and pose optimization. We further introduce a light field object tracking dataset containing challenging reflective objects with precise ground truth poses. Experiments demonstrate that our method is competitive with state-of-the-art model-based trackers in these difficult cases, paving the way toward universal object tracking in robotic systems. Code/data available at https://github.com/nagonch/LiFT-6DoF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。