用答案探测提升大模型解题多样性,避免陷入单一路径。
Answer Probing-Guided Search for Diverse Solution Exploration of LLMs

- 通过探测中间推理路径的答案隐藏状态来区分不同解法。
- 探测答案的困惑度可有效衡量推理正确性,指导搜索方向。
- 在多个任务上显著提升解法多样性,适合需要多方案的应用。
生成多个多样且高质量的解决方案对代码测试生成、药物发现等应用至关重要。然而,大语言模型在推理时往往收敛于单一高置信度解法,限制了对其他有效路径的探索。现有测试时方法通过树状搜索并使用响应级语义嵌入剪枝语义相似分支以促进多样性,但我们发现这些嵌入易受语言和风格相似性干扰,难以区分真正不同的解法路径。为此,我们提出答案探测(Answer Probing),即探测模型从中间推理路径可能得出的答案。实验证明,探测答案的隐藏状态比语义嵌入更有效地区分不同解法路径,且其困惑度可作为推理正确性的实用代理。基于此,我们提出答案探测引导的树搜索(APTS),利用探测答案的隐藏状态相似性和困惑度指导搜索。在两个大模型上的三个推理任务实验表明,APTS能持续提升解法多样性,验证了其有效性与鲁棒性。
原文摘要 · Abstract (English)
Generating multiple diverse and high-quality solutions is valuable for many applications, such as code-test generation and drug discovery. However, Large Language Models (LLMs) tend to converge on a single high-confidence solution during inference, limiting exploration of alternative valid solution paths. Existing test-time methods promote diversity through tree-like search and prune semantically similar branches using response-level semantic embeddings. However, we find that such embeddings are easily confounded by linguistic and stylistic similarities, making it difficult to distinguish genuinely distinct solution paths. To address this, we introduce Answer Probing, which probes the potential answer an LLM would reach from an intermediate reasoning path. We demonstrate that the hidden states of probed answers more effectively differentiate distinct solution paths than semantic embeddings, and the perplexity of probed answers serves as a practical proxy for reasoning correctness. Based on these findings, we propose Answer Probing-Guided Tree Search (APTS), which guides the tree search by the probed answers' hidden state similarity and perplexity. Experiments on three reasoning tasks across two LLMs show that APTS consistently enhances solution diversity, demonstrating its effectiveness and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。