让大模型自己判断何时该多试几步,省下大量计算却保持高准确率。
Learning When to Sample: Confidence-Aware Selective Sampling for Efficient Chain-of-Thought Reasoning
- 根据推理过程中的数值和语言特征,动态决定是否需要多路径采样
- 在多个数据集上减少71.7%的令牌消耗,准确率仅下降0.41个百分点
- 无需微调即可跨任务迁移,适合部署在资源受限场景
大型语言模型通过链式思维(CoT)推理可获得强性能,但常生成冗长路径导致高推理成本。基于自一致性的方法虽进一步提升准确率,却需采样并聚合多条推理路径,带来显著计算开销。本文提出一种置信度感知的选样框架,在推理时分析单条推理路径,自适应判断是否仅依赖该路径或触发多路径采样。框架利用推理状态中的轨迹级数值特征与句级语言特征引导选择性多路径推理。在MedQA上训练,并在MedQA、MathQA、MedMCQA和MMLU上进行域内及仅校准迁移评估,无需进一步微调。实验表明,该框架在保持与全量及高效多路径基线相当性能的同时,分别将令牌使用量降低71.7±5.0%和36.6±9.1%,准确率变化为-0.41±0.58和-0.31±0.58个百分点。结果表明,推理路径中蕴含丰富不确定性信号,可实现简单且可迁移的准确性与效率平衡机制。
原文摘要 · Abstract (English)
Large language models (LLMs) can achieve strong reasoning performance through chain-of-thought (CoT) reasoning, yet they often generate unnecessarily long reasoning paths that incur high inference cost. Self-consistency-based approaches push accuracy higher still, but they require sampling and aggregating multiple reasoning trajectories, leading to substantial computational overhead. In this paper, we introduce a confidence-aware selective sampling framework that, at inference time, analyzes a single reasoning trajectory to adaptively determine whether to rely on that trajectory alone or trigger multi-path sampling. The framework uses trajectory-level numeric features and sentence-level linguistic features extracted from reasoning states to guide selective multi-path reasoning. We train it on MedQA and evaluate it in-domain on MedQA and under calibration-only transfer on MathQA, MedMCQA, and MMLU, without further fine-tuning. Experimental results show that the proposed framework maintains comparable performance to full and efficient multi-path reasoning baselines, with accuracy changes of $-0.41 \pm 0.58$ and $-0.31 \pm 0.58$ percentage points, respectively, while reducing token usage by $71.7 \pm 5.0%$ and $36.6 \pm 9.1%$. These findings demonstrate that reasoning trajectories contain rich signals for uncertainty estimation, enabling a simple, transferable mechanism to balance accuracy and efficiency in LLM reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。