根据推理不确定性自动选大模型或小模型,既省资源又更准。
CP-Router: An Uncertainty-Aware Router Between LLM and LRM
- 用置信区间评估输出不确定性,动态决定调用大模型还是小模型。
- 在多个问答数据集上减少40%以上token消耗,准确率反而提升。
- 无需训练、适配任意模型对,适合追求效率与精度的部署场景。
大型推理模型(LRM)虽提升了长链推理能力,但对简单问题仍生成冗长输出,导致效率低下甚至准确率下降。为此,我们提出无需训练、通用的CP-Router路由框架,通过共形预测(Conformal Prediction, CP)估计预测不确定性,动态选择调用大语言模型(LLM)或推理模型(LRM),并在多项选择题问答(MCQA)任务中验证。为增强输入间不确定性区分度,引入全熵与二值熵(FBE)准则,自适应调整CP阈值。在涵盖数学、逻辑推理及中文化学等多类MCQA基准上的实验表明,相较于单独使用LRM,CP-Router可显著降低40%以上的token使用量,同时保持或提升准确率。该方法还可推广至多种模型组合与开放问答任务,展现出强泛化性与鲁棒性。
原文摘要 · Abstract (English)
Recent advances in Large Reasoning Models (LRMs) have significantly improved long-chain reasoning capabilities over Large Language Models (LLMs). However, LRMs often produce unnecessarily lengthy outputs even for simple queries, leading to inefficiencies or even accuracy degradation compared to LLMs. To overcome this, we propose CP-Router, a training-free and model-agnostic routing framework that dynamically selects between an LLM and an LRM, demonstrated with multiple-choice question answering (MCQA) prompts. The routing decision is guided by the prediction uncertainty estimates derived via Conformal Prediction (CP), which provides rigorous coverage guarantees. To further refine the uncertainty differentiation across inputs, we introduce Full and Binary Entropy (FBE), a novel entropy-based criterion that adaptively selects the appropriate CP threshold. Experiments across diverse MCQA benchmarks, including mathematics, logical reasoning, and Chinese chemistry, demonstrate that CP-Router efficiently reduces token usage while maintaining or even improving accuracy compared to using LRM alone. We also extend CP-Router to diverse model pairings and open-ended QA, where it continues to demonstrate strong performance, validating its generality and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。