评测大模型如何通过提问澄清模糊搜索，提升结果准确性。
Clarify-Then-Search: A Clarification Benchmark for Deep Search with End-to-End Nugget Restoration
- 构建真实搜索数据集，用问答澄清缺省信息以改进搜索
- 澄清后检索的准确率比不澄清高，3次提问效果最佳
- 发现模型常问无关区域问题，导致无效回答，影响整体性能
深度搜索在用户查询不明确时表现脆弱：缺少时间、地点、范围或定义等约束会导致检索偏差和答案不全。本文提出 Clarify-Then-Search 基准，评估大模型生成的澄清问题是否提升下游深度搜索效果。基于百度搜索引擎的真实查询数据，该基准包含 518 个精心筛选的实例，每个实例含一个意图查询与对应的不明确查询。对每个意图查询，运行 WebDancer 一次以归档证据，并构建带可追溯来源标识的加权证据金标准（golden reference） nuggets。评估时，澄清器（Clarifier）提出 k ∈ {1, 2, 3} 个问题；闭卷用户回答者（User Answerer）仅根据意图查询中明示信息作答，否则返回 unknown；闭卷重写器（Rewriter）仅使用原始不明确查询及问答对重写查询。WebDancer 在重写查询上执行，使用 restore_score_100（加权黄金片段召回率，支持部分得分）衡量端到端效用。所有模型在 k=1 时均优于无交互基线，预算越大收益越明显。GPT-5.2 在 k=1 时得分最高，而 ERNIE-4.5-Turbo-128K 在 k=3 时表现最佳。诊断发现系统普遍存在过度询问仅涉及区域的问题，这些通常无法从意图查询中回答，导致大量 unknown。该基准实现抗泄漏、可复现的澄清-搜索流程评估，支持对问题有效性、可回答性及预算影响的细粒度分析。
原文摘要 · Abstract (English)
Deep search is brittle on underspecified user queries: missing constraints such as time, location, scope, or definitions can lead to retrieval drift and incomplete answers. We introduce Clarify-Then-Search, a benchmark for evaluating whether LLM-generated clarification questions improve downstream deep-search utility. Built on real-world query data from the Baidu search engine, the benchmark contains 518 curated instances, each with an intent query and a corresponding underspecified query. For each intent query, we run WebDancer once to archive evidence and construct a static golden reference as weighted, evidence-grounded nuggets with traceable source identifiers. At evaluation time, a Clarifier asks k in {1, 2, 3} questions; a closed-book User Answerer replies only with information explicitly stated in the intent query, otherwise returning unknown; and a closed-book Rewriter produces a rewritten query using only the underspecified query and the elicited question-answer pairs. WebDancer then executes on the rewritten query, and we score end-to-end utility using restore_score_100, a weighted nugget-recall score with partial credit against the static gold. Across all evaluated models, clarification improves over the no-interaction baseline at k=1, and larger budgets generally yield further gains. GPT-5.2 achieves the highest mean score at k=1, while ERNIE-4.5-Turbo-128K becomes the overall top-performing model at k=3. Diagnostics reveal a consistent failure mode: many systems over-ask region-only questions that are often unanswerable from the intent and thus elicit unknown. Clarify-Then-Search enables leakage-resistant and reproducible evaluation of clarify-then-search pipelines, with fine-grained analyses of question utility, answerability, and budget effects in deep search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。