通过剪枝语义等价步骤，显著提升大模型搜索效率与准确率。
EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via Action Pruning
- 在推理搜索中识别并移除语义重复的操作步骤。
- 在GSM8K上使令牌消耗减少48.1%，同时提升准确率。
- 专为数学推理设计等价检测数据集MathEquiv，适合高效推理场景。
大型语言模型（LLM）在通过搜索算法进行复杂推理时表现优异，但现有策略常因对语义等价步骤的冗余探索导致大量令牌消耗。现有语义相似性方法在数学推理等专业领域难以准确识别等价性。为此，我们提出EquivPruner，一种简单有效的机制，在LLM推理搜索中识别并剪枝语义等价动作。我们还构建了首个数学命题等价数据集MathEquiv，用于训练轻量级等价检测器。在多种模型和任务上的实验表明，EquivPruner显著降低令牌消耗，提升搜索效率，并通常增强推理准确性。例如，在Qwen2.5-Math-7B-Instruct上应用于GSM8K时，令牌消耗减少48.1%，同时准确率提升。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel at complex reasoning through search algorithms, yet current strategies often suffer from massive token consumption due to redundant exploration of semantically equivalent steps. Existing semantic similarity methods struggle to accurately identify such equivalence in domain-specific contexts like mathematical reasoning. To address this, we propose EquivPruner, a simple yet effective approach that identifies and prunes semantically equivalent actions during LLM reasoning search. We also introduce MathEquiv, the first dataset we created for mathematical statement equivalence, which enables the training of a lightweight equivalence detector. Extensive experiments across various models and tasks demonstrate that EquivPruner significantly reduces token consumption, improving searching efficiency and often bolstering reasoning accuracy. For instance, when applied to Qwen2.5-Math-7B-Instruct on GSM8K, EquivPruner reduced token consumption by 48.1\% while also improving accuracy. Our code is available at https://github.com/Lolo1222/EquivPruner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。