用轻量方法实现大模型路由的低成本高精度选择
Cost-Aware Contrastive Routing for LLMs
- 将提示和模型映射到统一空间，快速匹配最优成本专家
- 在多个基准上提升准确率-成本权衡最高达25%
- 支持动态专家池变化，微秒级延迟适合实际部署
我们研究在多样且动态的模型池中对大语言模型进行成本感知路由。现有方法常忽略提示特异性上下文，依赖昂贵的模型剖析，假设专家集合固定，或采用低效的试错策略。本文提出轻量级框架成本谱对比路由（CSCR），将提示与模型映射至共享嵌入空间，实现快速、成本敏感的选择。CSCR对开源模型使用紧凑的快速计算逻辑值足迹，对黑盒API使用困惑度指纹。通过对比编码器训练，在自适应成本带内偏好最便宜且准确的专家。推理时仅需一次k-NN查找，基于FAISS索引，无需重训练即可适应专家池变化，实现微秒级延迟。在多个基准测试中，CSCR持续优于基线，准确率-成本权衡最高提升25%，且对未见大模型和分布外提示具有强泛化能力。
原文摘要 · Abstract (English)
We study cost-aware routing for large language models across diverse and dynamic pools of models. Existing approaches often overlook prompt-specific context, rely on expensive model profiling, assume a fixed set of experts, or use inefficient trial-and-error strategies. We introduce Cost-Spectrum Contrastive Routing (CSCR), a lightweight framework that maps both prompts and models into a shared embedding space to enable fast, cost-sensitive selection. CSCR uses compact, fast-to-compute logit footprints for open-source models and perplexity fingerprints for black-box APIs. A contrastive encoder is trained to favor the cheapest accurate expert within adaptive cost bands. At inference time, routing reduces to a single k-NN lookup via a FAISS index, requiring no retraining when the expert pool changes and enabling microsecond latency. Across multiple benchmarks, CSCR consistently outperforms baselines, improving the accuracy-cost tradeoff by up to 25%, while generalizing robustly to unseen LLMs and out-of-distribution prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。