构建可验证的旅行规划评估基准,检验智能体在杂乱网页中的真实推理能力。
VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

- 基于真实网页构建多模态检索库,强制智能体自主跨源搜索
- 通过逐单元验证机制量化事实可靠性,区分逻辑错误与幻觉
- 发现检索与推理存在权衡:自主检索会严重损害指令记忆
现有基准通过以API为中心的范式为旅行规划智能体奠定了基础。然而,随着自主智能体能力的提升,其评估必须从简单的工具执行转向应对开放网络的固有复杂性。当前基准忽略了信息噪声、多源事实矛盾,并未将视觉感知融入逻辑规划。我们提出VeriTrip,一个可验证的基准,以满足智能体鲁棒性和可靠性日益增长的需求。VeriTrip将评估重点转向对非结构化多模态网络语料的证据驱动推理。其基于真实来源建立的多模态检索基底(MRB),迫使智能体自主协调异构数据的查询。同步的可验证知识库(VKB)实现逐单元验证协议,精确量化事实可靠性,区分系统性推理失败与参数化幻觉。对主流多模态大模型的评估揭示了关键的‘检索-推理权衡’:自主检索的认知负荷显著削弱指令保持能力。VeriTrip为下一代能在无约束多模态环境中运行的规划智能体提供了严谨的基础。
原文摘要 · Abstract (English)
Existing benchmarks have laid the foundation for travel planning agents by establishing API-centric paradigms. However, as the capabilities of Autonomous Agents continue to advance, their evaluation must evolve beyond simple tool execution toward handling the inherent complexities of the open web. Current benchmarks bypass core cognitive hurdles: they fail to account for information noise, ignore multi-source factual contradictions, and overlook the necessity of grounding visual perception into logical planning. We introduce VeriTrip, a verifiable benchmark designed to meet the increasing demands for agent robustness and reliability. VeriTrip shifts the evaluation focus to evidence-grounded reasoning over unstructured multimodal web corpora. It establishes a Multimodal Retrieval Base (MRB) derived from real-world sources, forcing agents to autonomously orchestrate queries across heterogeneous data. A synchronized Verifiable Knowledge Base (VKB) enables a cell-wise verification protocol that precisely quantifies factual reliability, distinguishing systematic reasoning failures from parametric hallucinations. Our evaluations across leading MLLMs reveal a critical \textit{retrieval-reasoning trade-off}: the cognitive load of autonomous retrieval significantly erodes instruction retention. VeriTrip provides the rigorous foundation necessary for the next generation of planning agents capable of operating in unconstrained, multimodal environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。