HyDRA动态匹配查询需求与模型能力，实现低成本高效推理。
HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Pools
- 用多维度能力预测+缺额匹配选择最优模型
- 成本降低54.1%~72.5%，质量损失仅3.2点
- 无需重训，支持任意模型增删，跨语言通用
生产环境中的大模型部署正越来越多地采用异构模型池，其成本差异可达数量级。现有路由策略仅做强弱二元判断，且将学习参数绑定于具体模型，导致模型目录变更时需重新训练。我们提出HyDRA（混合动态路由架构），通过预测每个查询的细粒度多维能力需求，并基于配置定义的模型性能轮廓进行缺额匹配来选择最廉价满足要求的模型。采用ModernBERT编码器搭配K=4个独立sigmoid头，分别评估推理、代码生成、调试和工具使用能力；随后通过缺额匹配算法选取符合需求的最低成本模型。部署后的预测器在生产环境中平均CPU推理延迟为86毫秒，且完全解耦于模型目录——增删模型仅需配置更改，无需重训。在包含GPT-5.4-mini、Claude Haiku 4.5、GPT-5.3 Codex、Claude Sonnet 4.6、GPT-5.4的五模型池上，可调缺额阈值覆盖三种模式：峰值质量优于始终使用强力模型Claude Sonnet 4.6基线（75.4% vs. 74.2%解决率），成本节省12.9%；等质量下成本降低54.1%，相较此前内部二元路由提升6倍（原为9.1%）；激进模式下成本节省达72.5%，质量下降3.2点。结果在LiveCodeBench、BigCodeBench及tau-bench上具泛化性。HyDRA已部署于GitHub Copilot的VS Code Chat自动模式，首次在大模型路由文献中实现对中文、欧洲语系及其他文字系统的语言无关路由。
原文摘要 · Abstract (English)
Production LLM deployments increasingly maintain heterogeneous model pools spanning order-of-magnitude cost differences. Existing routers make binary strong-vs-weak decisions and couple learned parameters to specific model identities, requiring retraining whenever the catalog changes. We present HyDRA (Hybrid Dynamic Routing Architecture), a framework that predicts fine-grained, multi-dimensional capability requirements per query and matches them against configuration-defined model profiles via shortfall matching. A ModernBERT encoder with K=4 independent sigmoid heads scores each query along reasoning, code generation, debugging, and tool use; a shortfall-matching algorithm then selects the cheapest model whose capabilities meet the predicted requirements. The deployed predictor runs at 86 ms median CPU inference latency in production, and is fully decoupled from the model catalog -- adding or removing models requires only a configuration change, with zero retraining. On SWE-Bench Verified (5-model pool: GPT-5.4-mini, Claude Haiku 4.5, GPT-5.3 Codex, Claude Sonnet 4.6, GPT-5.4), HyDRA's tunable shortfall threshold spans three regimes: peak-quality exceeds the always-strong Claude Sonnet 4.6 baseline (75.4% vs. 74.2% resolution) at 12.9% cost savings; iso-quality matches Sonnet at 54.1% cost savings, a 6x improvement over our prior in-house binary router at 9.1%; aggressive pushes savings to 72.5% for a 3.2-point quality trade. Results generalize across LiveCodeBench, BigCodeBench, and tau-bench. HyDRA is deployed to all users in GitHub Copilot's VS Code Chat auto-mode and -- to our knowledge for the first time in the LLM routing literature -- demonstrates language-invariant routing across CJK, European, and other script families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。