用进化算法自动找最优RAG组合，跨领域提升问答质量。
RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
- 将RAG流程视为整体搜索空间，通过遗传算法优化9类技术组合。
- 在6个领域平均提升3.8%，最高达12.5%（检索）和7.5%（生成）。
- 发现向量检索+生成后反思是通用骨干，适合实际部署的RAG设计。
RAG性能受检索、排序、增强、提示和生成等多个环节交互影响，孤立优化模块易失效。本文提出RAGSmith，一个模块化框架，将RAG设计视为对九类技术家族及46,080种可行配置的端到端架构搜索。采用遗传搜索优化一个综合目标，联合评估检索指标（recall@k, mAP, nDCG, MRR）与生成指标（LLM-Judge和语义相似度）。在六个维基百科衍生领域（数学、法律、金融、医学、国防工业、计算机科学）上进行评估，每个领域包含100道涵盖事实型、解释型和长答案型的问题。RAGSmith找到的配置平均比基础RAG提升3.8%（各领域范围为+1.2%至+6.9%），检索最高提升12.5%，生成最高提升7.5%。搜索通常仅探索约0.2%的配置空间（约100个候选），发现一种稳健核心结构：向量检索结合生成后反思/修订，并辅以领域相关的扩展、重排序、增强和提示重排；段落压缩从未被选中。提升幅度与问题类型相关，事实型和长答案混合任务获益更大。结果为构建高效RAG系统提供实用、领域敏感的指导，并验证了进化搜索在全链路优化中的有效性。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) quality depends on many interacting choices across retrieval, ranking, augmentation, prompting, and generation, so optimizing modules in isolation is brittle. We introduce RAGSmith, a modular framework that treats RAG design as an end-to-end architecture search over nine technique families and 46{,}080 feasible pipeline configurations. A genetic search optimizes a scalar objective that jointly aggregates retrieval metrics (recall@k, mAP, nDCG, MRR) and generation metrics (LLM-Judge and semantic similarity). We evaluate on six Wikipedia-derived domains (Mathematics, Law, Finance, Medicine, Defense Industry, Computer Science), each with 100 questions spanning factual, interpretation, and long-answer types. RAGSmith finds configurations that consistently outperform naive RAG baseline by +3.8\% on average (range +1.2\% to +6.9\% across domains), with gains up to +12.5\% in retrieval and +7.5\% in generation. The search typically explores $\approx 0.2\%$ of the space ($\sim 100$ candidates) and discovers a robust backbone -- vector retrieval plus post-generation reflection/revision -- augmented by domain-dependent choices in expansion, reranking, augmentation, and prompt reordering; passage compression is never selected. Improvement magnitude correlates with question type, with larger gains on factual/long-answer mixes than interpretation-heavy sets. These results provide practical, domain-aware guidance for assembling effective RAG systems and demonstrate the utility of evolutionary search for full-pipeline optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。