仅用单目摄像头实现人体定位与轨迹预测，提升实际场景下的鲁棒性。
Unified Human Localization and Trajectory Prediction with Monocular Vision
- 基于Transformer的统一框架，联合优化定位与预测任务。
- 在标注数据上定位和预测准确率比基线高12%。
- 对噪声输入表现稳定，适合真实机器人应用。
传统人体轨迹预测模型依赖干净的标注数据，需专用设备或人工标注，难以用于机器人场景。现有方法在噪声输入下易过拟合，泛化能力差。本文提出MonoTransmotion（MT）框架，仅使用单目摄像头，联合解决人体定位与轨迹预测问题。该框架包含鸟瞰图（BEV）定位模块和轨迹预测模块：前者利用2D人体姿态估计位置，并引入新型方向性损失使序列定位更平滑；后者基于估计结果预测未来运动。通过联合训练，模型在噪声环境下更具鲁棒性。我们在标注数据集和真实非标注数据集上验证了模型性能：在标注数据集上，BEV定位与轨迹预测分别比基线提升约12%；在真实非标注数据集上，性能保持稳定，表明其强泛化能力。代码已开源。
原文摘要 · Abstract (English)
Conventional human trajectory prediction models rely on clean curated data, requiring specialized equipment or manual labeling, which is often impractical for robotic applications. The existing predictors tend to overfit to clean observation affecting their robustness when used with noisy inputs. In this work, we propose MonoTransmotion (MT), a Transformer-based framework that uses only a monocular camera to jointly solve localization and prediction tasks. Our framework has two main modules: Bird's Eye View (BEV) localization and trajectory prediction. The BEV localization module estimates the position of a person using 2D human poses, enhanced by a novel directional loss for smoother sequential localizations. The trajectory prediction module predicts future motion from these estimates. We show that by jointly training both tasks with our unified framework, our method is more robust in real-world scenarios made of noisy inputs. We validate our MT network on both curated and non-curated datasets. On the curated dataset, MT achieves around 12% improvement over baseline models on BEV localization and trajectory prediction. On real-world non-curated dataset, experimental results indicate that MT maintains similar performance levels, highlighting its robustness and generalization capability. The code is available at https://github.com/vita-epfl/MonoTransmotion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。