评测25个本地模型在机器人上的表现，找出兼顾速度、省电和教学效果的最佳方案。
Benchmarking Local Language Models for Social Robots using Edge Devices
- 在树莓派上测试25个开源模型，评估推理速度、能耗和教学能力。
- 模型性能差异大：吞吐量差10倍以上，知识准确率从随机到57.2%不等。
- 高知识准确率不是好教学的关键，适合资源受限的教育机器人部署。
面向社交教育机器人的互动式教学支持系统（如机器人学习伙伴RSC）需在计算资源严重受限的条件下实现响应迅速且保护隐私的交互。然而，针对教育场景下边缘设备上语言模型的系统性评测仍存在空白。本文对25个开源语言模型在边缘硬件上的本地部署进行了基准测试。评估涵盖三个维度：推理效率（每秒生成词元数、能耗）、通用知识（六类MMLU子集）及教学有效性（由大模型评分的教育质量），并在树莓派4（RPi4）为主平台，辅以RPi5和笔记本电脑GPU进行对比验证。结果表明显著权衡：不同模型间吞吐量与能效差异超一个数量级，MMLU准确率范围为近随机至57.2%，教学效果与任一指标均无单调相关性。其中Granite4 Tiny Hybrid（7B）表现最佳，达到2.5词元/秒、0.90词元/焦耳，以及54.6%的MMLU准确率；高知识准确率并非取得良好教学评分的必要条件。四款代表性模型的人类验证结果与自动排名高度一致（皮尔逊相关系数r=0.967，n=4）。基于此，我们提出一种三层次本地推理架构，用于在资源受限设备上平衡响应速度与准确性。
原文摘要 · Abstract (English)
Social-educational robots designed for socially interactive pedagogical support, such as the Robot Study Companion (RSC), rely on responsive, privacy-preserving interaction despite severely limited compute. However, there is a gap in systematic benchmarking of language models for edge computing in pedagogical applications. This paper benchmarks 25 open-source language models for local deployment on edge hardware. We evaluate each model across three dimensions: inference efficiency (tokens per second, energy consumption), general knowledge (a six-category MMLU subset), and teaching effectiveness (LLM-rated pedagogical quality), validated against five independent human raters using the Raspberry Pi(RPi)4 as the primary platform, with additional comparisons on the RPi5 and a laptop GPU. Results reveal pronounced trade-offs: throughput and energy efficiency vary by over an order of magnitude across models, MMLU accuracy ranges from near-random to 57.2%, and teaching effectiveness does not correlate monotonically with either metric. Among the evaluated models, Granite4 Tiny Hybrid (7B) achieves a strong overall balance, reaching 2.5 tokens per second, 0.90 tokens per joule, and 54.6% MMLU accuracy; high MMLU accuracy does not appear necessary for strong teaching scores. Human validation on four representative models preserved the automated rank ordering (Pearson r = 0.967, n = 4). Based on these findings, we propose a three-tier local inference architecture for the RSC that balances responsiveness and accuracy on resource-constrained hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。