用知识蒸馏让AI接收机更省电，性能还更好
Energy Efficiency in AI for 5G and Beyond: A DeepRx Case Study
- 用知识蒸馏训练小型AI接收机，模仿大模型表现
- 蒸馏后模型在不同信噪比下误码率更低，能效提升明显
- 适合5G/6G通信中对低功耗智能接收的场景
本研究聚焦于人工智能/机器学习模型在能源效率与性能间的平衡问题，以基于全卷积ResNet架构的DeepRX深度学习接收机为例。通过分析每瓦特浮点运算数（FLOPs/Watt）和每时钟周期浮点运算数（FLOPs/clock），发现能耗估算值与实测结果一致，且受内存访问模式影响显著。研究对比了训练与推理阶段的能耗动态。核心贡献在于应用知识蒸馏（KD）训练一个紧凑的深接收机学生模型，使其在保持教师模型性能的同时降低能耗。实验测试了不同学生模型规模、最优教师模型大小及蒸馏超参数。性能评估以误码率（BER）随信干噪比（SINR）变化为指标，结果显示蒸馏模型在各类SINR条件下均表现出更低的误码底限，验证了知识蒸馏在实现高效能人工智能方案中的有效性。
原文摘要 · Abstract (English)
This study addresses the challenge of balancing energy efficiency with performance in AI/ML models, focusing on DeepRX, a deep learning receiver based on a fully convolutional ResNet architecture. We evaluate the energy consumption of DeepRX, considering factors including FLOPs/Watt and FLOPs/clock, and find consistency between estimated and actual energy usage, influenced by memory access patterns. The research extends to comparing energy dynamics during training and inference phases. A key contribution is the application of knowledge distillation (KD) to train a compact DeepRX student model that emulates the performance of the teacher model but with reduced energy consumption. We experiment with different student model sizes, optimal teacher sizes, and KD hyperparameters. Performance is measured by comparing the Bit Error Rate (BER) performance versus Signal-to-Interference & Noise Ratio (SINR) values of the distilled model and a model trained from scratch. The distilled models demonstrate a lower error floor across SINR levels, highlighting the effectiveness of KD in achieving energy-efficient AI solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。