用指令路由让大模型统一处理20+视觉任务，无需重训练。
Olympus: A Universal Task Router for Computer Vision Tasks
- 通过指令控制将视觉任务分发给专用模块，实现灵活调度。
- 跨20项任务平均路由准确率94.75%，链式操作精度达91.82%。
- 适合希望扩展大模型视觉能力的研究者与开发者。
我们提出Olympus，一种将多模态大语言模型（MLLM）转化为统一视觉任务框架的新方法。通过控制器MLLM，Olympus将图像、视频和3D物体中的20余项专用任务分配给独立模块。基于指令的路由机制支持复杂工作流的链式执行，无需训练大型生成模型。Olympus可轻松集成至现有MLLM，扩展其能力且性能相当。实验表明，Olympus在20项任务上平均路由准确率达94.75%，链式操作场景下精度为91.82%，证明其作为通用任务路由器在多样化视觉任务中的有效性。项目页面：http://yuanze-lin.me/Olympus_page/
原文摘要 · Abstract (English)
We introduce Olympus, a new approach that transforms Multimodal Large Language Models (MLLMs) into a unified framework capable of handling a wide array of computer vision tasks. Utilizing a controller MLLM, Olympus delegates over 20 specialized tasks across images, videos, and 3D objects to dedicated modules. This instruction-based routing enables complex workflows through chained actions without the need for training heavy generative models. Olympus easily integrates with existing MLLMs, expanding their capabilities with comparable performance. Experimental results demonstrate that Olympus achieves an average routing accuracy of 94.75% across 20 tasks and precision of 91.82% in chained action scenarios, showcasing its effectiveness as a universal task router that can solve a diverse range of computer vision tasks. Project page: http://yuanze-lin.me/Olympus_page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。