梳理测试时扩展中的验证机制，揭示如何用无参数计算提升大模型推理性能。
Trust but Verify! A Survey on Verification Design for Test-time Scaling
- 统一归纳验证器的训练方法与类型，涵盖提示式与微调模型。
- 验证器通过评分候选输出，在解码空间中高效搜索最优结果。
- 适合关注推理增强、模型可信赖性研究者参考。
测试时扩展（TTS）已成为提升大语言模型性能的新前沿。通过在推理阶段使用更多计算资源，LLM 能够改进推理过程并提升任务表现。已有多种 TTS 方法，如从另一模型蒸馏推理路径，或利用验证器探索庞大的解码搜索空间。验证器作为奖励模型，对解码过程中的候选输出进行评分，以细致探索广阔解题空间并选择最优结果。该范式因推理时零参数扩展和显著性能提升而成为主流。验证器可为提示式、判别式或生成式模型，用于验证推理路径、结果或两者。尽管广泛应用，现有文献仍缺乏对多样化验证方法及其训练机制的系统梳理。本文综述了相关工作，提出验证器训练、类型及在测试时扩展中作用的统一视角。代码与资料库见：https://github.com/elixir-research-group/Verifierstesttimescaling.github.io。
原文摘要 · Abstract (English)
Test-time scaling (TTS) has emerged as a new frontier for scaling the performance of Large Language Models. In test-time scaling, by using more computational resources during inference, LLMs can improve their reasoning process and task performance. Several approaches have emerged for TTS such as distilling reasoning traces from another model or exploring the vast decoding search space by employing a verifier. The verifiers serve as reward models that help score the candidate outputs from the decoding process to diligently explore the vast solution space and select the best outcome. This paradigm commonly termed has emerged as a superior approach owing to parameter free scaling at inference time and high performance gains. The verifiers could be prompt-based, fine-tuned as a discriminative or generative model to verify process paths, outcomes or both. Despite their widespread adoption, there is no detailed collection, clear categorization and discussion of diverse verification approaches and their training mechanisms. In this survey, we cover the diverse approaches in the literature and present a unified view of verifier training, types and their utility in test-time scaling. Our repository can be found at https://github.com/elixir-research-group/Verifierstesttimescaling.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。