用距离加权提升大模型查询路由抗异常能力
ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers
- 基于输入与训练数据的距离加权，动态调整模型选择
- 在异常查询上准确率提升23%，普通查询性能不变
- 无需训练，适合资源受限的实时推理系统
大型语言模型（LLM）查询路由器对现代AI平台的效率至关重要，其通过将推理请求分配给准确且低成本的模型来优化性能。参数化路由器通常使用训练过的神经网络进行模型选择，但存在重训练和维护开销。非参数化路由器无需训练，通过输入查询与训练集查询编码间的相似性估算模型准确率和成本。然而，与参数化方法类似，它们在面对异常查询时泛化能力差，这一问题因训练集多样性有限而加剧——扩充训练集成本高，且难以跟上不断演进的应用场景。我们提出ProxRouter，采用指数倾斜聚合机制，在非参数化路由器中平衡偏差与方差，显著提升其对异常查询的鲁棒性。实验表明，ProxRouter在保持正常查询性能的同时，显著提升了异常查询的路由准确率，且开销极小。
原文摘要 · Abstract (English)
Large language model (LLM) query routers are critical to modern AI platforms as they seek to improve efficiency by assigning inference queries to accurate, yet low-cost models. Parametric routers typically use trained neural networks for LLM selection but suffer from retraining and maintenance overheads. Nonparametric routers are training-free, instead estimating LLM accuracy and cost via similarity between encodings of the input query and training set queries. However, like their parametric counterparts, nonparametric routers struggle to generalize to outlier queries, an issue exacerbated by limited diversity in training sets which are costly to expand and difficult to keep current with ever-evolving use cases. We propose ProxRouter, which applies an exponentially tilted aggregation mechanism to balance bias and variance in nonparametric routers, improving their robustness to outliers. Experiments show ProxRouter enhances outlier routing while preserving inlier performance with minimal overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。