通过可控嵌入探索提升大模型推理准确率
Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration
- 用嵌入扰动实现生成路径的受控搜索
- 结合贝叶斯优化，验证器引导下准确率显著提升
- 无需启发式策略，适合各类大模型快速部署
大语言模型在复杂推理任务中表现受限，主要源于生成路径多样性不足和搜索效率低下。本文提出Soft Reasoning，一种基于嵌入的搜索框架，通过优化首个词的嵌入来引导生成过程。该方法结合（1）嵌入扰动实现受控探索，以及（2）贝叶斯优化，利用验证器引导的目标函数不断精炼嵌入，平衡探索与利用。实验表明，该方法在极少计算开销下实现了更高的推理正确性与连贯性，且不依赖启发式搜索策略。结果证明其为可扩展、模型无关的高效解决方案。代码已开源：https://github.com/alickzhu/Soft-Reasoning。
原文摘要 · Abstract (English)
Large Language Models (LLMs) struggle with complex reasoning due to limited diversity and inefficient search. We propose Soft Reasoning, an embedding-based search framework that optimises the embedding of the first token to guide generation. It combines (1) embedding perturbation for controlled exploration and (2) Bayesian optimisation to refine embeddings via a verifier-guided objective, balancing exploration and exploitation. This approach improves reasoning accuracy and coherence while avoiding reliance on heuristic search. Experiments demonstrate superior correctness with minimal computation, making it a scalable, model-agnostic solution. The code is released at https://github.com/alickzhu/Soft-Reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。