arXiv:2506.14144cs.CVcs.AI2025-06被引 4

用视觉与大模型结合,让行人轨迹预测更符合真实环境约束。

SceneAware: Scene-Constrained Pedestrian Trajectory Prediction with LLM-Guided Walkability

  • 用ViT+MLLM解析场景,生成可走/不可走区域图
  • 在ETH/UCY数据集上比之前方法提升超50%
  • 适合机器人导航、安防系统等需要精准路径的场景

准确预测行人轨迹对机器人和监控系统至关重要。现有方法多关注行人间社交互动,常忽视显著影响移动模式的环境上下文。本文提出SceneAware,一种显式融合场景理解的轨迹预测框架。通过视觉变换器(ViT)编码器处理静态场景图像,多模态大语言模型(MLLM)在训练中生成二值可走性掩码,区分可达与受限区域。将基于Transformer的轨迹编码器与ViT场景编码器结合,同时捕捉时间动态与空间约束。引入碰撞惩罚机制,防止预测轨迹违反物理边界,确保结果合理。该框架包含确定性和随机性两种变体。在ETH/UCY基准数据集上的实验表明,本方法优于现有最先进模型,性能提升超过50%。不同轨迹类别分析显示,模型在各类行人运动中表现稳定。这凸显了显式使用场景信息的重要性,证明该场景感知方法在生成准确且物理合理的预测方面既有效又可靠。代码已公开:https://github.com/juho127/SceneAware。

原文摘要 · Abstract (English)

Accurate prediction of pedestrian trajectories is essential for applications in robotics and surveillance systems. While existing approaches primarily focus on social interactions between pedestrians, they often overlook the rich environmental context that significantly shapes human movement patterns. In this paper, we propose SceneAware, a novel framework that explicitly incorporates scene understanding to enhance trajectory prediction accuracy. Our method leverages a Vision Transformer~(ViT) scene encoder to process environmental context from static scene images, while Multi-modal Large Language Models~(MLLMs) generate binary walkability masks that distinguish between accessible and restricted areas during training. We combine a Transformer-based trajectory encoder with the ViT-based scene encoder, capturing both temporal dynamics and spatial constraints. The framework integrates collision penalty mechanisms that discourage predicted trajectories from violating physical boundaries, ensuring physically plausible predictions. SceneAware is implemented in both deterministic and stochastic variants. Comprehensive experiments on the ETH/UCY benchmark datasets show that our approach outperforms state-of-the-art methods, with more than 50\% improvement over previous models. Our analysis based on different trajectory categories shows that the model performs consistently well across various types of pedestrian movement. This highlights the importance of using explicit scene information and shows that our scene-aware approach is both effective and reliable in generating accurate and physically plausible predictions. Code is available at: https://github.com/juho127/SceneAware.

轨迹预测场景理解大模型应用机器人导航

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。