小模型在边缘设备上实现高效智能代理,突破云端依赖。
TinyLLM: Evaluation and Optimization of Small Language Models for Agentic Tasks on Edge Devices
- 用多种优化方法提升小模型的工具调用能力
- 1-3B模型达65.74%准确率,多轮任务55.62%准确率
- 适合隐私敏感、低延迟的边缘智能应用
本文研究小型语言模型(SLMs)在代理任务(函数/工具/API调用)中的有效性,聚焦于无需云基础设施即可在边缘设备运行智能体。通过伯克利函数调用排行榜(BFCL)框架评估了TinyAgent、TinyLlama、Qwen和xLAM等模型,涵盖简单、多重、并行、混合并行及相关性检测五类任务,测试了实时与非实时场景下的多轮表现。提出基于参数的优化策略,包括监督微调(SFT)、参数高效微调(PEFT)、强化学习(RL)、直接偏好优化(DPO)及混合方法。构建了基于AgentBank数据(如ALFRED)的DPO训练流程,利用TinyLlama生成拒绝样本并经人工验证。结果表明,中型模型(1-3B参数)显著优于超紧凑模型(<1B参数),整体准确率达65.74%,多轮任务准确率为55.62%,混合优化策略能有效提升小模型在边缘设备上的准确性、效率与稳定性,使隐私保护、低延迟的自主智能体在云端外成为可能。
原文摘要 · Abstract (English)
This paper investigates the effectiveness of small language models (SLMs) for agentic tasks (function/tool/API calling) with a focus on running agents on edge devices without reliance on cloud infrastructure. We evaluate SLMs using the Berkeley Function Calling Leaderboard (BFCL) framework and describe parameter-driven optimization strategies that include supervised fine-tuning (SFT), parameter-efficient fine-tuning (PEFT), reinforcement learning (RL)-based optimization, preference alignment via Direct Preference Optimization (DPO), and hybrid methods. We report results for models including TinyAgent, TinyLlama, Qwen, and xLAM across BFCL categories (simple, multiple, parallel, parallel-multiple, and relevance detection), both in live and non-live settings, and in multi-turn evaluations. We additionally detail a DPO training pipeline constructed from AgentBank data (e.g., ALFRED), including our conversion of SFT data to chosen-rejected pairs using TinyLlama responses as rejected outputs and manual validation. Our results demonstrate clear accuracy differences across model scales where medium-sized models (1-3B parameters) significantly outperform ultra-compact models (<1B parameters), achieving up to 65.74% overall accuracy, and 55.62% multi-turn accuracy with hybrid optimization. This study highlights the importance of hybrid optimization strategies that enable small language models to deliver accurate, efficient, and stable agentic AI on edge devices, making privacy-preserving, low-latency autonomous agents practical beyond the cloud.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。