根据置信度动态选模型，省钱又保准。
Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference
- 用置信度判断任务难易，小模型能搞定的就不动大模型
- 在MMLU上达大模型精度，算力省20%~40%
- 适合边缘设备和付费API场景，降低推理成本
大型语言模型（LLMs）在自然语言任务中显著提升了推理性能，但模型越大，计算开销越高。本文提出一种基于置信度的动态模型选择策略：通过评估模型对任务的答案把握程度及回复准确性，将高置信度任务交给小模型处理，低置信度或复杂任务则移交大模型，兼顾可靠性与计算效率。在Massive Multitask Language Understanding（MMLU）基准上的实验表明，该方法在保持与最大模型相当准确率的同时，计算成本降低了20%至40%。应用于GPT-4o API调用时，令牌消耗减少约60%，进一步提升成本效益。结果表明，基于置信度的模型选择策略在资源受限场景（如边缘设备和商业API应用）中具有重要实用价值。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have revolutionized inference across diverse natural language tasks, with larger models performing better but at higher computational costs. We propose a confidence-driven strategy that dynamically selects the most suitable model based on confidence estimates. By assessing a model's confidence in handling the task and response accuracy, tasks that are likely to be solved correctly are retained, while more uncertain or complex cases are delegated to a larger model, ensuring reliability while minimizing computation. Specifically, we evaluate a model's likelihood of knowing the correct answer and the probability that its response is accurate. Experiments on the Massive Multitask Language Understanding (MMLU) benchmark show that our approach achieves accuracy comparable to the largest model while reducing computational costs by 20\% to 40\%. When applied to GPT-4o API calls, it reduces token usage by approximately 60\%, further improving cost efficiency. These findings indicate the potential of confidence-based model selection to enhance real-world LLM deployment, particularly in resource-constrained settings such as edge devices and commercial API applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。