SATER让小模型更省钱更快，还能保持高精度。
SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading
- 用自知机制和信心拒绝对小模型做优化，减少无效计算。
- 在多个数据集上成本降低50%以上，级联延迟减少80%以上。
- 适合追求低成本高效率的AI应用开发人员使用。
大型语言模型（LLMs）在各类任务中表现优异，但其性能常依赖昂贵的商业API或云服务。模型选择需在性能与成本间权衡：高性能模型费用高昂，而低成本的小型语言模型（SLMs）能力有限。现有研究主要采用预生成路由和级联路由两种策略，前者快速但效率低，后者虽成本效益高但延迟大。为此，我们提出SATER，一种双模式兼容的方法，通过最短响应偏好优化和置信度感知拒绝机制对模型进行微调。该方法显著减少冗余输出与响应时间，同时提升预生成路由性能和级联路由效率。在三个SLMs和六个不同类型与复杂度的数据集上的实验表明，SATER在保持相当性能的同时，计算成本持续降低超过50%，级联延迟减少超过80%。
原文摘要 · Abstract (English)
Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness frequently depends on costly commercial APIs or cloud services. Model selection thus entails a critical trade-off between performance and cost: high-performing LLMs typically incur substantial expenses, whereas budget-friendly small language models (SLMs) are constrained by limited capabilities. Current research primarily proposes two routing strategies: pre-generation routing and cascade routing. Both approaches have distinct characteristics, with cascade routing typically offering superior cost-effectiveness and accuracy despite its higher latency. To further address the limitations of both approaches, we introduce SATER, a dual-mode compatible approach that fine-tunes models through shortest-response preference optimization and a confidence-aware rejection mechanism. SATER significantly reduces redundant outputs and response times, while improving both the performance of pre-generation routing and the efficiency of cascade routing. Experiments across three SLMs and six datasets, varying in type and complexity, demonstrate that SATER achieves comparable performance while consistently reducing computational costs by over 50\% and cascade latency by over 80\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。