多设备协同推理中动态调度，兼顾高精度与低延迟。
MultiTASC++: A Continuously Adaptive Scheduler for Edge-Based Multi-Device Cascade Inference
- 根据设备状态实时调整任务分流策略
- 在100台设备下保持目标满意度与最高精度
- 适合智能家庭等多样物联网场景
级联系统由轻量模型处理所有样本，再由更重的高精度模型对复杂样本进行优化，已成为移动和物联网设备实现高精度且低计算负担的分布式推理主流方法。随着智能家居等智能室内环境不断扩展，多设备级联新场景出现：多个异构设备同时共享服务器端的重型模型，该服务器通常位于或靠近用户环境。本文提出 MultiTASC++，一种持续自适应的多租户感知调度器，可动态调控设备的转发决策，以在保证高精度和低延迟的前提下最大化系统吞吐量。在多种设备环境及不同服务器模型下的大量实验表明，该调度器能持续维持目标满意度，并在不同设备层级和高达100台设备的负载下提供最高可用精度，验证了其在动态、多样化物联网环境中协同深度神经网络推理的可扩展性与高效性。
原文摘要 · Abstract (English)
Cascade systems, consisting of a lightweight model processing all samples and a heavier, high-accuracy model refining challenging samples, have become a widely-adopted distributed inference approach to achieving high accuracy and maintaining a low computational burden for mobile and IoT devices. As intelligent indoor environments, like smart homes, continue to expand, a new scenario emerges, the multi-device cascade. In this setting, multiple diverse devices simultaneously utilize a shared heavy model hosted on a server, often situated within or close to the consumer environment. This work introduces MultiTASC++, a continuously adaptive multi-tenancy-aware scheduler that dynamically controls the forwarding decision functions of devices to optimize system throughput while maintaining high accuracy and low latency. Through extensive experimentation in diverse device environments and with varying server-side models, we demonstrate the scheduler's efficacy in consistently maintaining a targeted satisfaction rate while providing the highest available accuracy across different device tiers and workloads of up to 100 devices. This demonstrates its scalability and efficiency in addressing the unique challenges of collaborative DNN inference in dynamic and diverse IoT environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。