不需训练，复用少量示例提升大模型输出准确性
Training-free LLM Verification via Recycling Few-shot Examples
- 复用输入的少量示例同时生成与验证答案
- 在7个任务上平均准确率提升8.2%
- 适合希望零成本提升模型可靠性的研究者
尽管大语言模型（LLMs）表现卓越，但其推理过程的固有随机性及结论不一致带来挑战。现有多数投票或引入外部验证器的方法存在适用性局限或需额外训练。为此，我们提出新框架ReFeri，通过复用少量示例来验证输出。核心思想是将给定示例用于生成与评估候选回答，结合前向置信度与后向重构惩罚，选出符合示例指导且避免演示特定过拟合的输出。在三个不同大模型、七个多样化任务上的实验表明，该框架显著提升模型准确率，平均相对增益达8.2%，实现有效响应选择。
原文摘要 · Abstract (English)
Although large language models (LLMs) have achieved remarkable performance, the inherent stochasticity of their reasoning processes and varying conclusions present significant challenges. Majority voting or Best-of-N with external verifiers has been explored to mitigate this, but these approaches are limited in applicability or require additional training. To address this problem, we propose a novel framework that Recycles Few-shot examples to verify LLM outputs (ReFeri). Our key idea is to utilize the given few-shot examples not only to generate outputs, but also to evaluate the candidate outputs. Specifically, ReFeri combines a forward confidence score with a backward reconstruction penalty to select candidates that follow few-shot guidance while avoiding demonstration-specific overfitting. Experiments with three different LLMs across seven diverse tasks demonstrate that our framework significantly improves the accuracy of LLMs---achieving an average relative gain of 8.2%---through effective response selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。