让大模型自动选快慢模式,看病问诊更省时省钱
SynapseRoute: An Auto-Route Switching Framework on Dual-State Large Language Model
- 根据问题难易自动切换思考或快速回答模式
- 准确率提升至0.8390,推理时间减少36.8%
- 适合医疗问答等对效率与成本敏感的应用
随着大语言模型在实际应用中的普及,选择合适模型需兼顾性能与运营成本。具备推理能力的模型使‘思考’(高推理)与‘非思考’(快速、低成本)模式间的成本差距进一步扩大。本工作发现,约58%的医学问题仅靠非思考模式即可准确回答,无需高成本推理过程。这揭示了问题复杂性的明显二分特征,表明基于复杂度动态路由查询至相应模式可优化准确性、成本效益与用户体验。据此,我们提出SynapseRoute——一种基于机器学习的动态路由框架,智能将输入查询分配至思考或非思考模式。在多个医学数据集上的实验表明,相比仅使用思考模式,SynapseRoute不仅整体准确率从0.8272提升至0.8390,且推理时间降低36.8%,令牌消耗减少39.66%。定性分析显示,对简单问题过度推理会导致延迟增加甚至准确率下降,而该方法有效避免了这一缺陷。最后,本文引入准确率-推理时间-令牌消耗(AIT)综合指标,全面评估三者权衡。
原文摘要 · Abstract (English)
With the widespread adoption of large language models (LLMs) in practical applications, selecting an appropriate model requires balancing not only performance but also operational cost. The emergence of reasoning-capable models has further widened the cost gap between "thinking" (high reasoning) and "non-thinking" (fast, low-cost) modes. In this work, we reveal that approximately 58% of medical questions can be accurately answered by the non-thinking mode alone, without requiring the high-cost reasoning process. This highlights a clear dichotomy in problem complexity and suggests that dynamically routing queries to the appropriate mode based on complexity could optimize accuracy, cost-efficiency, and overall user experience. Based on this, we further propose SynapseRoute, a machine learning-based dynamic routing framework that intelligently assigns input queries to either thinking or non-thinking modes. Experimental results on several medical datasets demonstrate that SynapseRoute not only improves overall accuracy (0.8390 vs. 0.8272) compared to the thinking mode alone but also reduces inference time by 36.8% and token consumption by 39.66%. Importantly, qualitative analysis indicates that over-reasoning on simpler queries can lead to unnecessary delays and even decreased accuracy, a pitfall avoided by our adaptive routing. Finally, this work further introduces the Accuracy-Inference-Token (AIT) index to comprehensively evaluate the trade-offs among accuracy, latency, and token cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。