用视觉地图反馈强化学习,让模型像人一样看图预测下一步位置。
Eyes Will Shut: A Vision-Based Next GPS Location Prediction Model by Reinforcement Learning from Visual Map Feed Back
- 将轨迹和道路网络转为图像,用视觉大模型进行类人推理。
- 在四个城市数据集上达到当前最优性能,跨城泛化能力更强。
- 适合做交通规划、导航推荐等需要理解人类移动习惯的场景。
下一位置预测是人类移动研究中的基础任务,广泛应用于交通规划、城市治理和疫情预测。现实中,人们常通过地图可视化轨迹,基于道路连通性和运动趋势进行推断。然而,大多数现有模型并未以类似人类的方式进行地图推理。得益于视觉语言模型(VLM)在视觉感知与推理方面的强大能力,我们提出一种新思路:将道路网络与轨迹渲染成图像,利用VLM的推理能力实现类人轨迹推断。首先提出视觉引导位置搜索(VGLS),验证通用VLM在不修改参数前提下具备轨迹推理潜力。基于此,进一步提出双阶段方法VLMLocPredictor:第一阶段设计两种监督微调任务,使VLM理解道路与轨迹结构并获得基本视觉推理能力;第二阶段引入基于视觉地图反馈的强化学习,让模型通过环境交互自我优化预测能力。在四个不同城市的实验中,该方法达到当前最优表现,并显著优于其他基于大模型的方法,在跨城市场景中展现出更强泛化性。
原文摘要 · Abstract (English)
Next Location Prediction is a fundamental task in the study of human mobility, with wide-ranging applications in transportation planning, urban governance, and epidemic forecasting. In practice, when humans attempt to predict the next location in a trajectory, they often visualize the trajectory on a map and reason based on road connectivity and movement trends. However, the vast majority of existing next-location prediction models do not reason over maps \textbf{in the way that humans do}. Fortunately, the recent development of Vision-Language Models (VLMs) has demonstrated strong capabilities in visual perception and even visual reasoning. This opens up a new possibility: by rendering both the road network and trajectory onto an image and leveraging the reasoning abilities of VLMs, we can enable models to perform trajectory inference in a human-like manner. To explore this idea, we first propose a method called Vision-Guided Location Search (VGLS), which evaluates whether a general-purpose VLM is capable of trajectory-based reasoning without modifying any of its internal parameters. Based on insights from the VGLS results, we further propose our main approach: VLMLocPredictor, which is composed of two stages: In the first stage, we design two Supervised Fine-Tuning (SFT) tasks that help the VLM understand road network and trajectory structures and acquire basic reasoning ability on such visual inputs. In the second stage, we introduce Reinforcement Learning from Visual Map Feedback, enabling the model to self-improve its next-location prediction ability through interaction with the environment. Experiments conducted on datasets from four different cities show that our method achieves state-of-the-art (SOTA) performance and exhibits superior cross-city generalization compared to other LLM-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。