让大模型自动决定何时停止多轮推理,既省成本又保证答案正确。
Adaptive Stopping for Multi-Turn LLM Reasoning
- 用分阶段误差预算实现多轮推理的自适应停止
- 在多个问答任务上达成目标覆盖率,同时减少推理轮次和成本
- 适合金融、医疗等对准确率要求高的高风险场景
大型语言模型(LLMs)越来越多地采用多轮推理与交互,如自适应检索增强生成(RAG)和ReAct风格智能体,以回答复杂问题。这些方法通过迭代式检索、推理或行动提升准确性,但面临核心挑战:何时应停止?现有方法依赖启发式规则或固定轮次预算,无法保证最终预测仍包含正确答案。这一局限在金融、医疗等高风险领域尤为严重,过早停止可能导致错误决策,过多轮次则增加成本与延迟。置信度预测(Conformal Prediction, CP)可提供形式化覆盖率保证,但现有LLM-CP方法仅适用于单次输出,不支持具有自适应停止能力的多轮流程。为此,我们提出首个针对多轮推理的CP框架——MiCP(Multi-Turn Language Models with Conformal Prediction)。MiCP在各轮次间分配不同误差预算,使模型能在保持整体覆盖率的前提下提前停止。我们在自适应RAG和ReAct上验证了该方法,在单跳与多跳问答基准上均达到目标覆盖率,同时显著降低轮次数、推理成本与预测集大小。此外,我们引入新指标,联合评估覆盖率有效性与回答效率。
原文摘要 · Abstract (English)
Large Language Models (LLMs) increasingly rely on multi-turn reasoning and interaction, such as adaptive retrieval-augmented generation (RAG) and ReAct-style agents, to answer difficult questions. These methods improve accuracy by iteratively retrieving information, reasoning, or acting, but introduce a key challenge: \textbf{When should the model stop?} Existing approaches rely on heuristic stopping rules or fixed turn budgets and provide no formal guarantees that the final prediction still contains the correct answer. This limitation is particularly problematic in high-stakes domains such as finance and healthcare, where unnecessary turns increase cost and latency, while stopping too early risks incorrect decisions. Conformal prediction (CP) provides formal coverage guarantees, but existing LLM-CP methods only apply to a single model output and cannot handle multi-turn pipelines with adaptive stopping. To address this gap, we propose Multi-Turn Language Models with Conformal Prediction (MiCP), the first CP framework for multi-turn reasoning. MiCP allocates different error budgets across turns, enabling the model to stop early while maintaining an overall coverage guarantee. We demonstrate MiCP on adaptive RAG and ReAct, where it achieves the target coverage on both single-hop and multi-hop question answering benchmarks while reducing the number of turns, inference cost, and prediction set size. We further introduce a new metric that jointly evaluates coverage validity and answering efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。