用多个验证器并行提升大模型推理能力，无需训练即可增强效果。
Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers
- 引入多验证器协同机制，通过多个独立模型评估输出不同方面。
- 实验表明，增加验证器数量可显著提升性能，优于自洽与奖励模型验证。
- 适合追求推理精度提升且无训练资源的开发者使用。
通过在测试阶段使用更多计算资源，大型语言模型（LLMs）可在不进行额外训练的情况下提升性能。一种常见策略是使用验证器评估候选输出。本文提出测试时计算的新扩展维度：扩大验证器的数量。我们引入多代理验证（MAV）作为测试时计算范式，结合多个验证器以提高性能。提出使用领域验证器（AVs），即预先训练好的LLM，通过提示词分别验证输出的不同方面，作为MAV系统中验证器的一种选择。由于无需额外训练即可组合，AVs是构建MAV的便捷模块。此外，我们提出BoN-MAV算法，将最佳n选一采样与多验证器结合。BoN-MAV展现出比自洽和奖励模型验证更强的扩展性，并证明了弱到强泛化（弱验证器可提升强模型）与自我改进（同一基模型既生成又验证）的能力。结果确立了扩大验证器数量作为提升语言模型测试阶段表现的有前景新方向。
原文摘要 · Abstract (English)
By utilizing more computational resources at test-time, large language models (LLMs) can improve without additional training. One common strategy uses verifiers to evaluate candidate outputs. In this work, we propose a novel scaling dimension for test-time compute: scaling the number of verifiers. We introduce Multi-Agent Verification (MAV) as a test-time compute paradigm that combines multiple verifiers to improve performance. We propose using Aspect Verifiers (AVs), off-the-shelf LLMs prompted to verify different aspects of outputs, as one possible choice for the verifiers in a MAV system. AVs are a convenient building block for MAV since they can be easily combined without additional training. Moreover, we introduce BoN-MAV, a simple multi-agent verification algorithm that combines best-of-n sampling with multiple verifiers. BoN-MAV demonstrates stronger scaling patterns than self-consistency and reward model verification, and we demonstrate both weak-to-strong generalization, where combining weak verifiers improves even stronger LLMs, and self-improvement, where the same base model is used to both generate and verify outputs. Our results establish scaling the number of verifiers as a promising new dimension for improving language model performance at test-time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。