用精选农业知识微调大模型,提升农技建议准确性与安全性。
Fine-Tuning and Evaluating Conversational AI for Agricultural Advisory
- 分离知识检索与对话生成,用专家标注的原子知识微调模型
- 在印度比哈尔邦测试中,事实召回率与F1值显著提升
- 小模型经微调后效果接近大模型,成本大幅降低
大型语言模型在农业咨询中展现出潜力,但原始模型常给出无依据建议、泛化且缺乏具体行动指引,沟通风格也不符合小农户需求。在关乎农民生计的高风险农业场景中,这些缺陷制约了负责任部署。本文提出一种混合式大模型架构:通过监督微调(LoRA)对专家标注的GOLDEN FACTS(原子化、经验证的农业知识单元)进行训练,优化事实召回;另设拼接层将检索到的知识转化为文化适配、安全敏感的自然回应。评估框架DG-EVAL基于专家标注的真实答案,而非维基或检索文档,实现原子级事实验证(衡量召回率、精确率与矛盾检测)。在印度比哈尔邦多作物、多问题的实验中,使用精选数据微调显著提升事实召回率与F1值,同时保持高相关性。采用微调后的较小模型,即可达到甚至超过前沿模型的事实质量,成本仅为后者的一小部分。拼接层进一步提升安全评分,同时维持高质量对话。论文发布farmerchat-prompts库,支持领域专用农业AI的可复现开发。
原文摘要 · Abstract (English)
Large Language Models show promise for agricultural advisory, yet vanilla models exhibit unsupported recommendations, generic advice lacking specific, actionable detail, and communication styles misaligned with smallholder farmer needs. In high stakes agricultural contexts, where recommendation accuracy has direct consequences for farmer outcomes, these limitations pose challenges for responsible deployment. We present a hybrid LLM architecture that decouples factual retrieval from conversational delivery: supervised fine-tuning with LoRA on expert-curated GOLDEN FACTS (atomic, verified units of agricultural knowledge) optimizes fact recall, while a separate stitching layer transforms retrieved facts into culturally appropriate, safety-aware responses. Our evaluation framework, DG-EVAL, performs atomic fact verification (measuring recall, precision, and contradiction detection) against expert-curated ground truth rather than Wikipedia or retrieved documents. Experiments across multiple model configurations on crops and queries from Bihar, India show that fine-tuning on curated data substantially improves fact recall and F1, while maintaining high relevance. Using a fine-tuned smaller model achieves comparable or better factual quality at a fraction of the cost of frontier models. A stitching layer further improves safety subscores while maintaining high conversational quality. We release the farmerchat-prompts library to enable reproducible development of domain-specific agricultural AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。