MapQaTor让地图问答数据标注效率提升30倍，支持多地图平台集成。
MapQaTor: An Extensible Framework for Efficient Annotation of Map-Based QA Datasets
- 基于API缓存与统一平台，实现地图问答数据的自动化采集与标注
- 标注速度比人工快至少30倍，保证数据一致性与可复现性
- 适合研究地理空间推理、LLM应用及地图数据集构建的开发者
地图导航服务如Google Maps、Apple Maps、OpenStreetMap在获取位置数据方面至关重要，但难以应对自然语言地理查询。尽管大语言模型（LLMs）在问答任务中展现潜力，但从地图服务构建可靠地理空间问答数据集仍具挑战。我们提出MapQaTor——一个可扩展的开源框架，简化可复现、可追溯的地图问答数据集创建流程。该框架支持任意地图API无缝接入，使用户能以极低配置从多元数据源收集并可视化信息。通过缓存API响应，平台确保地面实况一致，提升数据可靠性，即使真实世界信息变化亦然。MapQaTor将数据获取、标注与可视化整合于单一平台，为评估当前基于LLM的地理空间推理能力提供独特机会，并推动其地理理解能力发展。评估显示，相较于手动方法，该框架使标注速度提升至少30倍，凸显其在构建复杂地图推理数据集等地理空间资源方面的潜力。官网：https://mapqator.github.io/，演示视频：https://youtu.be/bVv7-NYRsTw。
原文摘要 · Abstract (English)
Mapping and navigation services like Google Maps, Apple Maps, OpenStreetMap, are essential for accessing various location-based data, yet they often struggle to handle natural language geospatial queries. Recent advancements in Large Language Models (LLMs) show promise in question answering (QA), but creating reliable geospatial QA datasets from map services remains challenging. We introduce MapQaTor, an extensible open-source framework that streamlines the creation of reproducible, traceable map-based QA datasets. MapQaTor enables seamless integration with any maps API, allowing users to gather and visualize data from diverse sources with minimal setup. By caching API responses, the platform ensures consistent ground truth, enhancing the reliability of the data even as real-world information evolves. MapQaTor centralizes data retrieval, annotation, and visualization within a single platform, offering a unique opportunity to evaluate the current state of LLM-based geospatial reasoning while advancing their capabilities for improved geospatial understanding. Evaluation metrics show that, MapQaTor speeds up the annotation process by at least 30 times compared to manual methods, underscoring its potential for developing geospatial resources, such as complex map reasoning datasets. The website is live at: https://mapqator.github.io/ and a demo video is available at: https://youtu.be/bVv7-NYRsTw.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。