用轻量级模型做智能路由,让虚拟世界无缝调用不同AI服务。
From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds

- 通过边缘端小模型解析用户指令意图,实现服务自动匹配。
- 微调后的小模型延迟低,能在中等硬件上运行整套流程。
- 适合想低成本接入多AI能力的虚拟世界开发者使用。
随着生成式AI能力扩展,驱动虚拟世界的架构面临挑战:用户通过多模态方式交互,但请求需要不同后台AI模型与算力。若将能力直接嵌入系统,会降低可扩展性、增加维护难度,并难以协调分布于边缘与云端的服务。本文提出一种基于小型语言模型(SLM)的代理编排网关,作为轻量级运行时协调机制,通过意图驱动的服务路由,解耦虚拟世界客户端与异构后端。边缘部署的SLM负责分类用户提示的语义意图,配置化的服务注册表验证并解析路由决策,选定后端透明调用,使新AI功能可无需修改客户端即可引入。该网关在InterwovenXR虚拟博物馆测试平台实现并评估。结果表明,紧凑的SLM可在边缘硬件上可靠充当意图路由器,任务特定微调可使参数低于十亿的模型成为实用、低延迟的路由器。采用细调后的子十亿参数模型作路由,搭配更大SLM生成对话响应的分层配置,可在中等边缘硬件上部署,效率优于单一模型承担全部任务。研究证明SLM可支持虚拟世界中的实际AI服务编排,贡献了一个可扩展、可拓展且支持边缘计算的架构,使虚拟代理成为访问分布式生成式AI服务的入口。
原文摘要 · Abstract (English)
As generative AI capabilities expand, AI-driven virtual worlds face a growing architectural challenge. Users interact through in-world interfaces in multimodal ways, yet their requests demand fundamentally different AI backend models and computational resources. Embedding these capabilities directly into virtual world systems reduces extensibility, complicates maintenance, and limits the ability to coordinate services distributed across edge and cloud infrastructure. This paper presents an SLM-based Agent Orchestration Gateway, a lightweight runtime coordination mechanism that decouples a virtual world client from heterogeneous AI backends through intent-driven service routing. An edge-deployed SLM classifies the semantic intent of each user prompt, a configurable service registry validates and resolves the routing decision, and the selected backend is invoked transparently, enabling new AI capabilities to be introduced in the virtual world without modifying the client application. The gateway is implemented and evaluated within the InterwovenXR virtual museum testbed. The evaluation shows that compact SLMs can serve as reliable intent routers on edge hardware, and that task-specific fine-tuning can transform sub-billion-parameter models into practical, low-latency routers. A layered configuration pairing a fine-tuned sub billion-parameter model as router with a larger SLM for conversational response generation is shown to be deployable on mid-range edge hardware and more efficient than delegating both responsibilities to a single model. The findings show that SLMs can support practical AI service orchestration in virtual worlds and the work contributes an evaluated architecture for scalable, extensible, and edge-supported AI interaction, enabling virtual agents become access points to distributed generative AI services.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。