用预填充激活值代替语义特征，实现更精准的模型路由。
LLM Router: Rethinking Routing with Prefill Activations
- 通过分离编码器与目标模型，利用激活值预测闭源模型表现。
- 最佳模型比独立模型接近最优水平45.58%差距，节省74.31%成本。
- 适合需要高效调用多个大模型的系统开发者参考。
现有路由机制依赖语义查询特征或人工设计特征，难以捕捉模型特异性失败或任务内在难度。本文改用大型语言模型内部激活值，特别是残差流（residual stream）作为路由信号。核心思想是编码器-目标解耦：将生成预测信号的编码器与需评估正确性的目标模型分离，使开源编码器可预测闭源模型性能。通过分层几何探针发现费希尔可分性（J）能有效识别关键层，辅以有效维度（d_eff）诊断验证。随后采用共享主干网络（SharedTrunkNet），一个联合多输出MLP，基于拼接的预填充特征同时预测多个候选模型的正确性概率。实验表明，SharedTrunkNet持续优于语义基线；最优情况下，其缩小了最强独立模型与理想调度器之间45.58%的差距，且相较最昂贵模型节省74.31%开销。结果证明，预填充激活值提供了稳健的路由信号，确立了基于激活值的路由为纯语义选择的高性能替代方案。
原文摘要 · Abstract (English)
Existing routers rely on semantic query features or handcrafted features, which often fail to capture model-specific failures or intrinsic task difficulty. We instead route using internal LLM activations, specifically the residual stream. Our key idea, Encoder-Target Decoupling, separates the model that produces the predictive signal (the Encoder) from the model whose correctness is being estimated (the Target), allowing open-weight encoders to predict the performance of closed-source target models. We evaluate layerwise geometric probes, finding that Fisher Separability ($J$) effectively identifies informative layers, supported by Effective Dimensionality ($d_{\mathrm{eff}}$) diagnostics. We then utilize a SharedTrunkNet, a joint multi-output MLP that predicts simultaneous correctness probabilities across candidate models using concatenated prefill features. In our experiments, SharedTrunkNet consistently outperforms semantic baselines. At its best, SharedTrunkNet closes 45.58% of the gap between the strongest standalone model and the oracle while achieving 74.31% cost savings relative to the most expensive model. These results demonstrate that prefill activations provide a robust routing signal, establishing activation-based routing as a high-performance alternative to purely semantic selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。