通过区分策略使用与可执行性,提升数学推理中的引导效果。
Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance
- 基于人类与模型解题差异,识别策略可执行性问题
- 提出选择性策略检索框架,实现跨源策略组合优化
- 在多个基准上提升准确率,最高达+13点
基于示例的引导广泛用于提升数学推理性能,但其效果在不同问题和模型间极不稳定——即使引导正确且相关。我们发现这种不稳定性源于一个此前被忽视的差距:策略使用(是否出现在成功解法中)与策略可执行性(作为引导时对目标模型是否有效)之间的分离。通过对人工与模型生成解法的对照分析,我们发现人类与模型策略在结构上存在系统性差异,导致互补优势及源依赖性反转现象。基于此诊断,我们提出测试时框架选择性策略检索(SSR),通过多路径、源感知的实证信号,显式建模策略可执行性,有选择地检索并组合策略。在多个数学推理基准上,SSR显著优于直接求解、上下文学习及单源引导,在AIME25上准确率最高提升+13点,在Apex上提升+5点,适用于紧凑型推理模型。代码与数据集已公开于:https://github.com/lwd17/strategy-execute-pipeline。
原文摘要 · Abstract (English)
Example-based guidance is widely used to improve mathematical reasoning at inference time, yet its effectiveness is highly unstable across problems and models-even when the guidance is correct and problem-relevant. We show that this instability arises from a previously underexplored gap between strategy usage-whether a reasoning strategy appears in successful solutions-and strategy executability-whether the strategy remains effective when instantiated as guidance for a target model. Through a controlled analysis of paired human-written and model-generated solutions, we identify a systematic dissociation between usage and executability: human- and model-derived strategies differ in structured, domain-dependent ways, leading to complementary strengths and consistent source-dependent reversals under guidance. Building on this diagnosis, we propose Selective Strategy Retrieval (SSR), a test-time framework that explicitly models executability by selectively retrieving and combining strategies using empirical, multi-route, source-aware signals. Across multiple mathematical reasoning benchmarks, SSR yields reliable and consistent improvements over direct solving, in-context learning, and single-source guidance, improving accuracy by up to $+13$ points on AIME25 and $+5$ points on Apex for compact reasoning models. Code and benchmark are publicly available at: https://github.com/lwd17/strategy-execute-pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。