arXiv:2604.01235cs.AI2026-04

不同大模型后端下,结构化路由的最优方案差异显著。

Runtime Burden Allocation for Structured LLM Routing in Agentic Expert Systems: A Full-Factorial Cross-Backend Methodology

  • 从系统级负担分配角度重构结构化路由问题
  • 48种配置下测试发现无通用最优模式
  • 适合在多后端部署中优化性能的团队

结构化LLM路由常被当作提示工程问题处理。我们主张其本质上是系统级负担分配问题。随着大语言模型成为智能体系统的核心控制组件,可靠的结构化路由必须在真实部署约束下平衡正确性、延迟与实现成本。我们发现,这一平衡不仅受提示或模式影响,更取决于生成栈中结构化工作如何分配:是模型直接输出结构、传输时压缩,还是生成后本地重建。通过覆盖OpenAI、Gemini和Llama后端的48种部署配置及15,552次请求的全因子基准测试,我们得出核心结论:不存在通用最优路由模式。后端特异性交互效应主导性能表现。某些在Gemini和OpenAI上可靠的模式在Llama上会显著降低正确性,而压缩实现带来的效率提升也强烈依赖后端。本文不提供孤立模型对比,而是提出可在异构后端条件下应用的可部署推理框架,包含跨后端评估方法与生产级智能体专家系统的实用部署指导。

原文摘要 · Abstract (English)

Structured LLM routing is often treated as a prompt-engineering problem. We argue that it is, more fundamentally, a systems-level burden-allocation problem. As large language models (LLMs) become core control components in agentic AI systems, reliable structured routing must balance correctness, latency, and implementation cost under real deployment constraints. We show that this balance is shaped not only by prompts or schemas, but also by how structural work is allocated across the generation stack: whether output structure is emitted directly by the model, compressed during transport, or reconstructed locally after generation. We evaluate this formulation through a comprehensive full-factorial benchmark covering 48 deployment configurations and 15,552 requests across OpenAI, Gemini, and Llama backends. Our central finding is consequential: there is no universal best routing mode. Instead, backend-specific interaction effects dominate performance. Modes that remain highly reliable on Gemini and OpenAI can suffer substantial correctness degradation on Llama, while efficiency gains from compressed realization are strongly backend-dependent. Rather than presenting another isolated model comparison, this work contributes a deployable framework for reasoning about structured routing under heterogeneous backend conditions. We provide a cross-backend evaluation methodology and practical deployment guidance for navigating the correctness-cost-latency frontier in production-grade agentic expert systems.

LLM路由系统优化多后端智能体系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。