Dora让边缘AI推理更快更省电,还能保证用户体验。
Dora: QoE-Aware Hybrid Parallelism for Distributed Edge AI
- 根据设备差异自动拆分模型,生成符合体验要求的执行方案。
- 在动态网络中优化计算与通信重叠,提速1.1至6.3倍。
- 适合资源受限的智能家居、交通分析等边缘部署场景。
随着边缘AI应用普及,满足用户质量体验(QoE)要求(如推理延迟)已成为首要目标,因这些模型运行在资源受限设备上且直接与用户交互。然而,现代AI模型常超出单个设备的资源能力,需在异构设备间跨不稳定、易拥塞的网络进行分布式执行。现有混合并行(如数据并行与流水线并行)规划多以吞吐量或设备利用率为目标,忽视QoE,导致严重资源浪费(如不必要的能耗)或运行时体验违规。本文提出Dora框架,实现分布式边缘AI训练与推理中的QoE感知混合并行。Dora通过三个关键机制联合优化异构计算、拥塞网络与多维QoE目标:(i) 异构感知模型分区器,确定并分配模型各部分到设备,生成一组符合QoE约束的执行计划;(ii) 拥塞感知网络调度器,通过最大化计算与通信重叠来进一步优化候选计划;(iii) 运行时自适应器,动态组合多个计划,在保障全局效率的同时维持整体QoE。在智能家庭、交通分析及小型边缘集群等典型部署中,Dora实现1.1–6.3倍加速,或降低21%–82%能耗,同时在运行时动态下保持稳定体验。
原文摘要 · Abstract (English)
With the proliferation of edge AI applications, satisfying user quality of experience (QoE) requirements, such as model inference latency, has become a first class objective, as these models operate in resource constrained settings and directly interact with users. Yet, modern AI models routinely exceed the resource capacity of individual devices, necessitating distributed execution across heterogeneous devices over variable and contention prone networks. Existing planners for hybrid (e.g., data and pipeline) parallelism largely optimize for throughput or device utilization, overlooking QoE, leading to severe resource inefficiency (e.g., unnecessary energy drain) or QoE violations under runtime dynamics. We present Dora, a framework for QoE aware hybrid parallelism in distributed edge AI training and inference. Dora jointly optimizes heterogeneous computation, contention prone networks, and multi dimensional QoE objectives via three key mechanisms: (i) a heterogeneity aware model partitioner that determines and assigns model partitions across devices, forming a compact set of QoE compliant plans; (ii) a contention aware network scheduler that further refines these candidate plans by maximizing compute communication overlap; and (iii) a runtime adapter that adaptively composes multiple plans to maximize global efficiency while respecting overall QoEs. Across representative edge deployments, including smart homes, traffic analytics, and small edge clusters, Dora achieves 1.1--6.3 times faster execution and, alternatively, reduces energy consumption by 21--82 percent, all while maintaining QoE under runtime dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。