arXiv:2510.14660cs.CLcs.AI2025-10被引 5

用结构化评分标准提升搜索增强型大模型的验证效率与鲁棒性。

An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs

  • 将信息点拆解为可验证的评分条目,统一评估不同任务。
  • 在长文本任务中自动构建多条评分标准,支持动态网页内容。
  • 40亿参数的生成式验证器,训练高效且准确率高,适合实际部署。

搜索增强使大语言模型具备检索能力,突破静态参数限制。现有基于强化学习的奖励建模存在局限:规则奖励(如精确匹配)对表达变化敏感,难以用于长文本;生成式奖励虽更鲁棒,但在动态语料库中设计可验证、稳定的奖励仍具挑战,且计算成本高。本文提出统一可验证的「信息点即评分标准」范式,将原子信息点作为不同搜索增强任务的结构化评估依据。短文本任务对应单一评分标准,长文本任务则扩展为多个与问题需求对齐的评分标准。为此,我们设计基于查询重写的自动评分标准构建流程,可从静态语料和动态网络内容中自动提取相关段落并提炼评分标准。进一步提出 extbf{Search-Gen-V},一个40亿参数的高效生成式验证器,采用蒸馏思想和两阶段训练策略。实验表明,Search-Gen-V 在多种工作负载下均实现高验证准确率,是可扩展、鲁棒且高效的可验证奖励构造器。

原文摘要 · Abstract (English)

Search augmentation empowers Large Language Models with retrieval capabilities to overcome the limitations imposed by static parameters. Recently, Reinforcement Learning leverages tailored reward signals as a viable technique to enhance LLMs performing tasks involving search. However, existing reward modeling for search-augmented LLMs faces several limitations. Rule-based rewards, such as Exact Match, are verifiable but fragile to variations in expression and cannot be applied to long-form workloads. In contrast, generative rewards improve robustness, but designing verifiable and stable rewards for long-form workloads in dynamic corpora remains challenging and also incurs high computational costs. In this paper, we propose a unified and verifiable paradigm, "nugget-as-rubric", which treats atomic information points as structured evaluation criteria for different search-augmentation workloads. Short-form tasks correspond to a single rubric, whereas long-form tasks expand to multiple rubrics aligned with the question's information needs. To support long-form settings, we design an automatic rubric construction pipeline based on query rewriting, which can automatically retrieve passages relevant to each question and extract rubrics from them, both from static corpora and from dynamic online web content. Furthermore, we introduce \textbf{Search-Gen-V}, a 4B-parameter efficient generative verifier under our proposed verifiable paradigm, which is trained via the idea of distillation and a two-stage strategy. Experimental results show that Search-Gen-V achieves strong verification accuracy across different workloads, making it a scalable, robust, and efficient verifiable reward constructor for search-augmented LLMs.

大模型验证搜索增强生成式奖励评分标准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。