用小型模型在农场本地运行智能决策系统,兼顾隐私与效率
Evaluating Small Language Models for Agentic On-Farm Decision Support Systems
- 构建五类任务代理的智能系统,支持农场本地部署
- Qwen-4B在20个模型中表现最佳,但对数据库交互不稳定
- 首次评估小模型在奶牛养殖决策中的可行性,适合资源受限场景
大语言模型虽有潜力辅助奶农和学者决策、提升非技术群体的知识获取,但其高算力需求使其几乎只能通过云端服务使用,难以应用于实际牧场。为此,需开发可在本地农场硬件运行的轻量级替代方案。本文在真实农场计算条件下,评测了HuggingFace上20个开源小语言模型(SLM)。基于前期工作,构建了一个集成五类任务代理的智能系统:文献检索、网络搜索、SQL数据库交互、NoSQL数据库交互及基于预测模型的图表生成。评估分两阶段进行:第一阶段用5个测试问题筛选出能基本执行奶业指令且在资源受限环境下稳定的模型;第二阶段对通过筛选的模型用30个问题(每类任务5个,另含1个诚信与不当行为类别)进一步评估。结果表明,Qwen-4B在多数任务中表现最优,但在通过PySpark进行的NoSQL数据库交互中表现不稳定。据我们所知,这是首个明确评估小语言模型作为奶业决策支持系统引擎可行性的研究,重点强调隐私保护与计算效率。尽管结果显示小模型在奶业实践中具有应用前景,但挑战依然存在,仍需微调以提升其在特定奶业问题上的表现。
原文摘要 · Abstract (English)
Large Language Models (LLM) hold potential to support dairy scholars and farmers by supporting decision-making and broadening access to knowledge for stakeholders with limited technical expertise. However, the substantial computational demand restricts access to LLM almost exclusively through cloud-based service, which makes LLM-based decision support tools impractical for dairy farming. To address this gap, lightweight alternatives capable of running locally on farm hardware are required. In this work, we benchmarked 20 open-source Small Language Models (SLM) available on HuggingFace under farm-realistic computing constraints. Building on our prior work, we developed an agentic AI system that integrates five task-specific agents: literature search, web search, SQL database interaction, NoSQL database interaction, and graph generation following predictive models. Evaluation was conducted in two phases. In the first phase, five test questions were used for the initial screening to identify models capable of following basic dairy-related instructions and performing reliably in a compute-constrained environment. Models that passed this preliminary stage were then evaluated using 30 questions (five per task category mentioned above, plus one category addressing integrity and misconduct) in phase two. In results, Qwen-4B achieved superior performance across most of task categories, although showed unstable effectiveness in NoSQL database interactions through PySpark. To our knowledge, this is the first work explicitly evaluating the feasibility of SLM as engines for dairy farming decision-making, with central emphases on privacy and computational efficiency. While results highlight the promise of SLM-assisted tools for practical deployment in dairy farming, challenges remain, and fine-tuning is still needed to refine SLM performance in dairy-specific questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。