首个面向本地生活服务的智能搜索基准，测试大模型多跳推理能力
LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services
- 构建涵盖130万商户的跨城市多领域搜索基准
- 顶尖模型在多跳问答上正确率仅35.60%，完整性和忠实性不足
- 适合研究本地服务智能代理、多源推理的开发者与学者
大型推理模型（LRMs）推动了智能搜索系统在多源信息中进行复杂多步推理的能力。然而，现有研究多集中于通用信息检索，较少关注具有独特挑战的垂直领域。本文聚焦本地生活服务，提出首个综合性基准LocalSearchBench，包含覆盖6个服务类别、9个主要城市的130万商户数据，以及900个来自真实用户查询的多跳问答任务，需跨商家和产品进行多步推理。我们还开发了LocalPlayground，一个集成多种工具的统一交互环境。实验表明，即使最先进的模型（DeepSeek-V3.2）在该基准上的正确率也仅为35.60%，平均完整性仅60.32%，忠实性低至30.72%。这凸显了本地生活服务领域专用基准与针对性训练的重要性。代码、基准与排行榜已公开：https://localsearchbench.github.io/。
原文摘要 · Abstract (English)
Recent advances in large reasoning models LRMs have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explores vertical domains with unique challenges. In this work, we focus on local life services and introduce LocalSearchBench, which encompass diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, LocalSearchBench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed LocalPlayground, a unified environment integrating multiple tools for LRMs interaction. Experiments show that even state-of-the-art LRMs struggle on LocalSearchBench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services. Code, Benchmark, and Leaderboard are available at https://localsearchbench.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。