6G时代下，多层级协同推理系统降低AI计算延迟。
Distributed Collaborative Inference System in Next-Generation Networks and Communication
- 按网络层级分配不同大小模型，实现资源优化部署。
- 任务卸载策略使推理延迟降低17%，精度不降。
- 改进早退出机制，提升单模型推理效率，适合边缘计算场景。
随着人工智能的快速发展，生成式人工智能（GAI）正引领数据处理方式的变革。然而，GAI的高计算需求对资源受限设备构成挑战。迈向第六代移动网络（6G）的过程中，更高的数据速率和更优的能效要求GAI具备更高效的数据处理能力。传统GAI难以满足这些需求。为此，我们提出一种面向下一代网络与通信的多层级协同推理系统。该系统通过将不同规模的模型部署于网络各层级，并设计任务卸载策略以优化效率与延迟；同时引入改进的早退出机制，增强单模型推理性能。实验结果表明，相比现有方法，本系统在保持推理精度的前提下，最多可将推理时间缩短17%。
原文摘要 · Abstract (English)
With the rapid advancement of artificial intelligence, generative artificial intelligence (GAI) has taken a leading role in transforming data processing methods. However, the high computational demands of GAI present challenges for devices with limited resources. As we move towards the sixth generation of mobile networks (6G), the higher data rates and improved energy efficiency of 6G create a need for more efficient data processing in GAI. Traditional GAI, however, shows its limitations in meeting these demands. To address these challenges, we introduce a multi-level collaborative inference system designed for next-generation networks and communication. Our proposed system features a deployment strategy that assigns models of varying sizes to devices at different network layers. Then, we design a task offloading strategy to optimise both efficiency and latency. Furthermore, a modified early exit mechanism is implemented to enhance the inference process for single models. Experimental results demonstrate that our system effectively reduces inference latency while maintaining high-quality output. Specifically, compared to existing work, our system can reduce inference time by up to 17% without sacrificing the inference accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。