arXiv:2603.20223cs.CYcs.AI2026-03

比较边缘与云端模型在教育AI中的能耗与延迟,发现量化能省电但增时,影响教学效果。

Inference Energy and Latency in AI-Mediated Education: A Learning-per-Watt Analysis of Edge and Cloud Models

  • 用4比特量化压缩模型,降低推理能耗
  • 4比特量化每轮耗电329焦耳,比全精度低10.8%
  • 适合关注能耗和部署成本的教育AI开发者

即时反馈是有效AI辅助学习的核心需求,但其能量与延迟成本尚未充分研究。本研究通过实证比较微软Phi-3 Mini(4k-instruct)在NVIDIA T4 GPU上的两种本地推理配置:全精度FP16与4比特NormalFloat(NF4)量化,在支持KV缓存的500个跨五大学科的中学教育提示下进行评估。生成的1000条回答由10位剑桥国际教师与3个前沿AI系统组成的混合小组,依据四维评分标准进行教学质量评估。提出‘每瓦学习量’(LpW)新指标,衡量单位能量内学习价值。实际部署中,NF4每推理耗电329焦耳,低于FP16的369焦耳,但延迟从9.2秒增至13.4秒,使FP16在LpW上仍具1.33倍优势,质量差异为0.19分。若禁用缓存(仅用于离线评估),差距扩大至7.4倍,严重夸大FP16优势。量化效率依赖硬件与推理模式,对低资源地区公平部署有重要影响。

原文摘要 · Abstract (English)

Immediate feedback is a foundational requirement of effective AI-mediated learning, yet the energy and latency costs of delivering it remain largely unexamined. This study investigates the latency-energy-learning trade-off in AI tutoring through an empirical comparison of two on-device inference configurations of Microsoft Phi-3 Mini (4k-instruct) on an NVIDIA T4 GPU: full-precision FP16 and 4-bit NormalFloat (NF4) quantisation. Both were evaluated under KV-cache-enabled inference across 500 educational prompts spanning five secondary school subject domains. Pedagogical quality was assessed for each of the 1000 generated responses by a hybrid panel of 10 Cambridge International teachers and three frontier AI systems using a four-dimension rubric. We introduce Learning-per-Watt (LpW), a novel metric quantifying pedagogical value per unit of energy over the learner's waiting window. Under realistic deployment, NF4 achieves lower per-inference energy than FP16 (329 J vs. 369 J) but higher latency (13.4 s vs. 9.2 s), yielding a modest FP16 advantage in LpW of 1.33x at a quality difference of 0.19 points. Under cache-disabled inference -- used in offline evaluation but absent from real deployments -- the gap widens to 7.4x, overstating the FP16 advantage by more than fivefold. Quantisation efficiency is hardware-dependent and inference-regime dependent, with significant implications for equitable AI tutoring deployment in low-resource settings.

AI教育能耗分析量化推理边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。