不依赖标注数据，用模型自检提升推理准确性
Score $\times$ Decoder: A Unified View of Unsupervised Inference-Time Scaling for Hallucination Mitigation
- 用模型自我验证+虚拟思考提示，实现无监督的推理纠错
- 在MATH500上，自检得分配合优化解码可显著减少幻觉
- 模型能力与解码方式需协同选择，无万能方案
大语言模型即使答案在其参数中仍会幻觉。尽管推理时缩放能挖掘潜在知识，但最有效的方法需要监督：训练过的验证器或奖励模型。我们探讨仅用基础语言模型能做什么：哪种内在信号最能识别正确输出，以及如何解码？将此问题建模为得分×解码器的组合网格，涵盖四种得分（困惑度、对比度、幂分布似然、自验证）与三种解码方法（优化、采样、共识），在MATH500数据集上评估了基础版和指令微调版Qwen3-1.7B的表现。结果显示，自验证（通过提示模型自我判断答案并使用免训练的虚拟思考前缀增强）在多数设置下表现良好；但无固定最优得分——其效果取决于所使用的解码器及模型能力。当无监督可用时，得分与解码族必须共同选择。
原文摘要 · Abstract (English)
Large language models hallucinate even when the answer lies within their parameters. While inference-time scaling can surface this latent knowledge, the most effective methods require supervision: a trained verifier or reward model. We ask what can be done with only a base language model: which intrinsic signal best identifies correct outputs, and how should it be decoded? We cast this as a score~$\times$~decoder grid pairing four scores (perplexity, contrastive, power-distribution likelihood, and self-verification) with three decoding families (optimization, sampling, consensus), and evaluate every cell on MATH500 with the base and instruction-tuned Qwen3-1.7B. While self-verification, which prompts the model to judge its own answer and is sharpened by a training-free virtual-thinking prefix, works well in most settings, no score has a fixed quality: its value depends on the decoder that consumes it and on model capability. When no supervision is available, the score and the decoding family must be chosen together.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。