根据输入难易动态选轻模型，提速降耗不丢精度
PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution
- 用机器学习调度器从多个预训练模型中选最轻合适者
- 在CIFAR、TinyImageNet等数据集上减少36%计算量
- 适合部署在资源受限设备上的实时推理场景
深度神经网络在计算机视觉、语音识别和机器人等领域广泛应用，但大模型往往计算开销大、能耗高。由于这类代价仅在处理复杂输入时必要，对简单输入采用更轻量模型可显著提升效率而几乎不影响准确率。我们提出PERTINENCE，一种运行时方法：从一组预训练模型中，为每个输入选择最轻且可能正确处理它的模型。该方法使用基于机器学习的调度器完成选择，并通过遗传算法探索调度器训练策略，以找到准确率与计算成本之间的帕累托最优平衡。我们在针对CIFAR-10和CIFAR-100训练的CNN、TinyImageNet上的ViT，以及基于YOLO的真实交通路口摄像头数据的道路占用估计应用上评估了PERTINENCE。结果表明，其准确率与当前最优预训练模型相当或更高，同时将计算操作减少最多达36%，并通过可调的调用间隔实现等效或更低的端到端推理时间。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) are widely used for their ability to model complex patterns across domains such as computer vision, speech recognition, and robotics. However, larger models, while often more accurate, are computationally expensive and energy-intensive. Since such a cost is typically needed only for challenging inputs, dynamically selecting lighter models for simpler inputs can improve efficiency with minimal impact on accuracy. We introduce PERTINENCE, a runtime method that selects, from a set of pre-trained models, the lightest model likely to process each input correctly. An ML-based dispatcher performs this selection, and a genetic algorithm explores dispatcher training strategies to identify Pareto-optimal trade-offs between accuracy and computational cost. We evaluate PERTINENCE on CNNs trained on CIFAR-10 and CIFAR-100, ViTs trained on TinyImageNet, and a YOLO-based road occupancy estimation application using real-time intersection camera feeds. Results show that PERTINENCE matches or improves the accuracy of state-of-the-art pre-trained models while reducing operations by up to 36%, with equivalent or lower end-to-end inference time through tunable invocation intervals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。