构建真实旅行规划数据集，评估大模型生成行程的实用性。
TripTailor: A Real-World Benchmark for Personalized Travel Planning
- 基于超50万真实景点和近4000条行程构建基准测试集。
- 顶尖大模型生成行程中不足10%达到人类水平表现。
- 聚焦行程可行性、合理性与个性化，适合智能旅行代理研究者使用。
大型语言模型（LLMs）推理能力的持续提升使其在复杂任务中扮演重要角色，尤其在个性化高质量行程规划需求增长的背景下。然而，现有基准多依赖不真实的模拟数据，无法反映大模型生成行程与真实行程的差异。现有评估指标主要关注约束满足，难以全面衡量行程整体质量。为此，我们提出TripTailor，一个专为真实世界个性化旅行规划设计的基准。该数据集包含超过50万条真实世界的兴趣点（POIs）和近4,000条多样化的旅行行程，每条行程均附带详细信息，提供更真实的评估环境。实验表明，当前最先进的大模型生成的行程中，少于10%能达到人类水平。我们还识别出行程可行性、合理性和个性化定制等关键挑战。希望TripTailor能推动能够理解用户需求并生成实用行程的旅行规划智能体的发展。代码与数据集已开源：https://github.com/swxkfm/TripTailor。
原文摘要 · Abstract (English)
The continuous evolution and enhanced reasoning capabilities of large language models (LLMs) have elevated their role in complex tasks, notably in travel planning, where demand for personalized, high-quality itineraries is rising. However, current benchmarks often rely on unrealistic simulated data, failing to reflect the differences between LLM-generated and real-world itineraries. Existing evaluation metrics, which primarily emphasize constraints, fall short of providing a comprehensive assessment of the overall quality of travel plans. To address these limitations, we introduce TripTailor, a benchmark designed specifically for personalized travel planning in real-world scenarios. This dataset features an extensive collection of over 500,000 real-world points of interest (POIs) and nearly 4,000 diverse travel itineraries, complete with detailed information, providing a more authentic evaluation framework. Experiments show that fewer than 10\% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. Moreover, we identify several critical challenges in travel planning, including the feasibility, rationality, and personalized customization of the proposed solutions. We hope that TripTailor will drive the development of travel planning agents capable of understanding and meeting user needs while generating practical itineraries. Our code and dataset are available at https://github.com/swxkfm/TripTailor
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。