用聊天机器人让普通人也能轻松分析公交数据做决策
Design and testing of an agent chatbot supporting decision making with public transport data
- 基于智能体架构，让大模型调用查询、绘图等工具完成复杂任务
- 通过真实数据库生成测试用例，评估回答一致性与查询正确性
- 适合交通管理者或数据初学者快速获取公交服务分析结果
评估公共交通服务质量需分析大量关于计划与实际行程的数据，以及满足各项质量约束的文档。使用SQL查询、整理和可视化这些数据对多数用户而言十分复杂。本文提出一个聊天机器人，提供友好的交互方式来支持决策。该系统基于智能体架构，通过一系列工具扩展了核心大语言模型（LLM）的能力，可执行如运行SQL查询、绘制图表及根据行程坐标生成地图等任务。本文还解决了生成式AI项目中的关键难题：如何收集数据以衡量系统性能。聊天机器人通过一套工作流进行了充分测试，该流程包含多个问题，每个问题均记录生成的查询语句、检索到的数据及自然语言回复。这些问题源自基础示例，并结合数据库中的实际数据进行填充。此过程生成了一个可用于评估聊天机器人性能的数据集，尤其关注回答的一致性和查询的正确性。
原文摘要 · Abstract (English)
Assessing the quality of public transportation services requires the analysis of large quantities of data on the scheduled and actual trips and documents listing the quality constraints each service needs to meet. Interrogating such datasets with SQL queries, organizing and visualizing the data can be quite complex for most users. This paper presents a chatbot offering a user-friendly tool to interact with these datasets and support decision making. It is based on an agent architecture, which expands the capabilities of the core Large Language Model (LLM) by allowing it to interact with a series of tools that can execute several tasks, like performing SQL queries, plotting data and creating maps from the coordinates of a trip and its stops. This paper also tackles one of the main open problems of such Generative AI projects: collecting data to measure the system's performance. Our chatbot has been extensively tested with a workflow that asks several questions and stores the generated query, the retrieved data and the natural language response for each of them. Such questions are drawn from a set of base examples which are then completed with actual data from the database. This procedure yields a dataset for the evaluation of the chatbot's performance, especially the consistency of its answers and the correctness of the generated queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。