对比RAG与长文本大模型在农业技术问答中的表现，发现混合检索增强更优。
Agri-Query: A Case Study on RAG vs. Long-Context LLMs for Cross-Lingual Technical Question Answering
- 用混合检索增强策略提升跨语言技术问答效果
- 使用RAG后模型准确率超85%，优于直接提示长文本模型
- 适合工业领域技术文档问答研究者参考
我们开展了一项案例研究，评估具有128K令牌上下文窗口的大语言模型（LLMs）在技术问答（QA）任务中的表现。基准测试基于一份提供英语、法语和德语版本的农业机械用户手册，模拟跨语言信息检索场景：英文提问，需在三种语言的手册中查找答案。评估聚焦于真实的“大海捞针”挑战，并包含无法回答的问题以检验幻觉现象。我们比较了九种长上下文LLM直接提示的表现与三种检索增强生成（RAG）策略（关键词、语义、混合）的效果，采用大模型作为裁判进行评估。结果显示，在该特定手册上，混合RAG始终优于直接长上下文提示。Gemini 2.5 Flash和较小的Qwen 2.5 7B模型在使用RAG时，跨所有语言均达到超过85%的准确率。本文贡献了对工业领域专用语言模型性能的详细分析，以及一个可复用的开放评估框架，揭示了实际应用中的权衡与挑战。
原文摘要 · Abstract (English)
We present a case study evaluating large language models (LLMs) with 128K-token context windows on a technical question answering (QA) task. Our benchmark is built on a user manual for an agricultural machine, available in English, French, and German. It simulates a cross-lingual information retrieval scenario where questions are posed in English against all three language versions of the manual. The evaluation focuses on realistic "needle-in-a-haystack" challenges and includes unanswerable questions to test for hallucinations. We compare nine long-context LLMs using direct prompting against three Retrieval-Augmented Generation (RAG) strategies (keyword, semantic, hybrid), with an LLM-as-a-judge for evaluation. Our findings for this specific manual show that Hybrid RAG consistently outperforms direct long-context prompting. Models like Gemini 2.5 Flash and the smaller Qwen 2.5 7B achieve high accuracy (over 85%) across all languages with RAG. This paper contributes a detailed analysis of LLM performance in a specialized industrial domain and an open framework for similar evaluations, highlighting practical trade-offs and challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。