用信号驱动的路由框架,智能选择多模态大模型的最优推理路径。
vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models
- 通过13种异构信号组合成可配置的路由规则,实现灵活决策。
- 支持多云、隐私合规、成本优化等不同部署场景,统一架构适配。
- 集成防幻觉检测与个性化记忆系统,保障安全与上下文连贯性。
随着大语言模型在模态、能力与成本上的多样化,推理时为每个请求智能选择合适模型已成为关键系统挑战。本文提出vLLM语义路由器,一种面向多模态混合模型(MoM)部署的信号驱动决策路由框架。该架构基于香农信息论与布尔代数双重视角:信息论层面通过信号提取降低‘选哪个模型’的不确定性;布尔代数层面则通过信号条件组合构成功能完备的路由策略。核心创新在于可组合的信号编排:13类异构信号(包括亚毫秒级启发式与神经分类器,覆盖语义、安全、模态等)通过可配置布尔规则组合成特定部署的路由策略,使多云企业、隐私监管、成本优化等场景仅通过配置即可适配同一架构。匹配决策通过13种选择算法驱动模型路由,并引入插件链强制执行安全约束,包括三阶段HaluGate幻觉检测与轻量级回忆记忆系统ReflectionGate以支持个性化多轮对话。采用类型化的神经符号领域特定语言(DSL)定义策略并编译至多种部署目标,实现无需代码修改的配置优先适应。整体表明,可组合信号编排使单一框架能服务于多样化的成本、隐私与安全需求。
原文摘要 · Abstract (English)
As large language models (LLMs) diversify across modalities, capabilities, and cost profiles, the problem of intelligent request routing: selecting the right model for each query at inference time, has become a critical systems challenge. We present vLLM Semantic Router, a signal-driven decision routing framework for Mixture-of-Modality (MoM) model deployments. The architecture follows two complementary Shannon-inspired views. In the information-theoretic regime, signal extraction reduces the entropy of "which model?" by distilling routing-relevant information from raw queries. In the Boolean-algebraic regime, the decision engine composes functionally complete routing policies from signal conditions. The central innovation is composable signal orchestration: thirteen heterogeneous signal types, spanning sub-millisecond heuristics and neural classifiers for semantics, safety, and modality, are composed through configurable Boolean decision rules into deployment-specific routing policies, so that fundamentally different scenarios (multi-cloud enterprise, privacy-regulated, cost-optimized) are expressed as different configurations over the same architecture. Matched decisions drive semantic model routing via thirteen selection algorithms, while per-decision plugin chains enforce safety constraints including a three-stage HaluGate hallucination detection pipeline and a lightweight episodic memory system with ReflectionGate for personalized multi-turn context. A typed neural-symbolic DSL specifies these routing policies and compiles them to multiple deployment targets, enabling configuration-first adaptation without code changes. Together, these components show that composable signal orchestration enables a single framework to serve diverse deployment scenarios with differentiated cost, privacy, and safety policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。