通过视觉定位投影实现遮挡目标的轨迹去噪与预测,提升自动驾驶安全性。
Out-of-Sight Embodied Agents: Multimodal Tracking, Sensor Fusion, and Trajectory Forecasting
- 利用相机标定建立视觉与位置对应关系,无监督去除传感器噪声。
- 在Vi-Fi和JRDB数据集上实现当前最优去噪与轨迹预测性能。
- 首次将视觉-定位投影用于遮挡目标去噪,适用于自动驾驶与机器人场景。
轨迹预测是计算机视觉、视觉-语言-动作模型、世界模型及自主系统中的基础问题,对自动驾驶、机器人和监控具有广泛影响。然而,现有方法多假设观测完整且干净,难以处理因视野受限、遮挡或缺乏真实去噪轨迹导致的遮挡目标与噪声传感信号。这带来安全风险并降低实际部署鲁棒性。本文在先前工作基础上,将出视野轨迹(OST)任务从行人扩展至行人与车辆,增强其在自动驾驶、机器人与监控中的适用性。提出改进的视觉-定位去噪模块,利用相机标定建立视觉与位置对应关系,缓解视觉线索缺失问题,实现对噪声传感器信号的有效无监督去噪。在Vi-Fi与JRDB数据集上的大量实验表明,本方法在轨迹去噪与预测上均达到当前最优效果,显著优于基线方法。还对比了经典去噪方法(如卡尔曼滤波),并适配了近期轨迹预测模型,建立了更强基准。据我们所知,这是首个使用视觉-定位投影对遮挡目标的噪声轨迹进行去噪的工作,为未来研究开辟新方向。
原文摘要 · Abstract (English)
Trajectory prediction is a fundamental problem in computer vision, vision-language-action models, world models, and autonomous systems, with broad impact on autonomous driving, robotics, and surveillance. However, most existing methods assume complete and clean observations, and therefore do not adequately handle out-of-sight agents or noisy sensing signals caused by limited camera coverage, occlusions, and the absence of ground-truth denoised trajectories. These challenges raise safety concerns and reduce robustness in real-world deployment. In this extended study, we introduce major improvements to Out-of-Sight Trajectory (OST), a task for predicting noise-free visual trajectories of out-of-sight objects from noisy sensor observations. Building on our prior work, we expand Out-of-Sight Trajectory Prediction (OOSTraj) from pedestrians to both pedestrians and vehicles, increasing its relevance to autonomous driving, robotics, and surveillance. Our improved Vision-Positioning Denoising Module exploits camera calibration to establish vision-position correspondence, mitigating the lack of direct visual cues and enabling effective unsupervised denoising of noisy sensor signals. Extensive experiments on the Vi-Fi and JRDB datasets show that our method achieves state-of-the-art results for both trajectory denoising and trajectory prediction, with clear gains over prior baselines. We also compare with classical denoising methods, including Kalman filtering, and adapt recent trajectory prediction models to this setting, establishing a stronger benchmark. To the best of our knowledge, this is the first work to use vision-positioning projection to denoise noisy sensor trajectories of out-of-sight agents, opening new directions for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。