动态分配计算资源，让AI模型在速度与精度间自由切换。
ORXE: Orchestrating Experts for Dynamically Configurable Efficiency
- 用多个预训练专家构建可动态调整的推理路径。
- 实测在多数场景下优于单个专家和其他动态模型。
- 无需重新训练，适合部署在多种设备上。
本文提出ORXE，一种模块化且可适应的框架，实现AI模型的实时可配置效率。通过利用一组计算成本和性能水平各异的预训练专家，ORXE根据输入样本的复杂度动态调整推理路径。与需要复杂元模型训练的传统方法不同，ORXE在不增加开发复杂度的前提下实现高效率与灵活性。系统采用基于置信度的门控机制，为每个输入分配合适的计算资源，并支持运行时在推理开销与预测性能之间进行广泛调整。我们实现了无需训练的ORXE系统，用于图像分类任务，在多种设备上评估其效率与准确率。结果表明，ORXE在大多数情况下均优于单一专家及其他动态模型。该方法可扩展至其他应用，为多样化的实际部署场景提供可扩展解决方案。
原文摘要 · Abstract (English)
This paper presents ORXE, a modular and adaptable framework for achieving real-time configurable efficiency in AI models. By leveraging a collection of pre-trained experts with diverse computational costs and performance levels, ORXE dynamically adjusts inference pathways based on the complexity of input samples. Unlike conventional approaches that require complex metamodel training, ORXE achieves high efficiency and flexibility without complicating the development process. The proposed system utilizes a confidence-based gating mechanism to allocate appropriate computational resources for each input. ORXE also supports adjustments to the preference between inference cost and prediction performance across a wide range during runtime. We implemented a training-free ORXE system for image classification tasks, evaluating its efficiency and accuracy across various devices. The results demonstrate that ORXE achieves superior performance compared to individual experts and other dynamic models in most cases. This approach can be extended to other applications, providing a scalable solution for diverse real-world deployment scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。