通过分解查询任务本质，避免模型记忆陷阱，提升LLM路由泛化能力。
Beyond Query Memorization: Large Language Model Routing with Query Decomposition and Historical Matching
- 将查询路由转化为历史相似查询匹配，摆脱表面特征依赖。
- 在分布内与分布外数据上均保持高准确率，推理成本显著降低。
- 适合追求高效、鲁棒LLM部署的研究者与工程师使用。
优化大语言模型（LLM）部署中预测性能与计算成本的权衡是核心挑战。现有路由方法主要基于查询表层特征直接映射到模型，易陷入记忆陷阱，导致对分布外（OOD）数据泛化能力差。本文提出DecoR框架，将路由任务重构为从历史日志中筛选相似查询的匹配过程，有效缓解记忆陷阱。为提升匹配精度，引入查询能力解构方法，将语言表层形式与任务内在需求解耦，引导匹配聚焦于能力维度，使决策基于关键任务属性。此外，构建CodaSet基准用于评估路由泛化能力，实验表明DecoR在分布内与分布外场景下均保持高精度且显著降低推理开销。代码与数据已开源：https://github.com/lvbotenbest/DecoR。
原文摘要 · Abstract (English)
Optimizing the trade-off among predictive performance and computational cost is a central focus in the deployment of Large Language Models (LLMs). Current routing methods primarily rely on direct mapping from queries to models based on surface-level features, making them susceptible to the memorization trap and leading to poor generalizability on out-of-distribution (OOD) data. In this paper, we propose DecoR, a novel routing framework that recasts the routing task as a matching process of sifting similar queries from historical logs, effectively mitigating the memorization trap. To enhance matching accuracy, we introduce a query capability deconstruction method that decouples linguistic surface forms from task-intrinsic requirements, directing matching toward capability dimensions to ground decisions in essential task attributes. Furthermore, we develop CodaSet, a comprehensive benchmark for assessing routing generalization, where experimental results demonstrate that DecoR maintains superior accuracy while substantially lowering inference costs across both in-distribution and OOD settings. All the codes and data are available at https://github.com/lvbotenbest/DecoR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。