一个智能调度器协调多种模态工具，让AI自动完成复杂查询。
One Supervisor, Many Modalities: Adaptive Tool Orchestration for Autonomous Queries
- 用中央调度器动态拆解任务，按需分配文本、图像等专用工具。
- 相比传统方法，答对时间减少72%，对话返工降85%，成本降67%。
- 适合需要多模态协同的智能助手、自动化系统开发者使用。
我们提出一种自主多模态查询处理的智能体框架，通过中央调度器协调文本、图像、音频、视频和文档等多种模态的专用工具。该调度器动态分解用户请求，将子任务委派给适配的工具（如目标检测、OCR、语音转写），并采用自适应路由策略整合结果，而非依赖预设决策树。对于纯文本查询，使用基于学习的路由（RouteLLM）；非文本路径则借助小语言模型（SLM）实现模态分解。在15类任务的2,847个查询上评估，本框架相较匹配的分层基线，实现了72%的答对时间降低、85%的对话返工减少和67%的成本下降，同时保持准确率相当。结果表明，智能集中式编排显著提升了多模态AI部署的经济性。
原文摘要 · Abstract (English)
We present an agentic AI framework for autonomous multimodal query processing that coordinates specialized tools across text, image, audio, video, and document modalities. A central Supervisor dynamically decomposes user queries, delegates subtasks to modality-appropriate tools (e.g., object detection, OCR, speech transcription), and synthesizes results through adaptive routing strategies rather than predetermined decision trees. For text-only queries, the framework uses learned routing via RouteLLM, while non-text paths use SLM-assisted modality decomposition. Evaluated on 2,847 queries across 15 task categories, our framework achieves 72% reduction in time-to-accurate-answer, 85% reduction in conversational rework, and 67% cost reduction compared to the matched hierarchical baseline while maintaining accuracy parity. These results demonstrate that intelligent centralized orchestration fundamentally improves multimodal AI deployment economics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。