分析RAG系统超参数对速度与准确率的影响,帮开发者平衡性能与成本。
Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency
- 对比不同向量库、分块策略和重排序方法的效率与效果
- 使用6项指标评估,发现Faiss精度更高但更慢,小窗口分块最快
- 适合需要高召回率的场景,如医疗决策支持系统
大型语言模型虽表现优异,但常出现幻觉或依赖过时知识。检索增强生成(RAG)通过结合外部搜索弥补这一缺陷。本文分析超参数对RAG系统速度与质量的影响,涵盖Chroma与Faiss向量存储、分块策略、交叉编码器重排序及温度设置,并评估六项指标:忠实性、答案正确性、相关性、上下文精确率、上下文召回率和答案相似性。Chroma查询速度快13%,但Faiss检索精确率更高,揭示了明显的速度-精度权衡。简单固定长度分块(小窗口、低重叠)在保持快速的同时优于语义分割。重排序带来小幅质量提升,但运行时间增加约5倍,其价值取决于延迟要求。这些结果帮助实践者在计算成本与准确性间做出权衡。最后,我们以修正型RAG工作流重新评估最优配置,发现其优势在模型可迭代请求额外证据时依然成立。最终达到近乎完美的上下文精确率(99%),表明通过合理调参,RAG系统可实现极高检索准确率,这对检索质量直接影响下游任务表现的应用(如医疗辅助决策)具有重要意义。
原文摘要 · Abstract (English)
Large language models achieve high task performance yet often hallucinate or rely on outdated knowledge. Retrieval-augmented generation (RAG) addresses these gaps by coupling generation with external search. We analyse how hyperparameters influence speed and quality in RAG systems, covering Chroma and Faiss vector stores, chunking policies, cross-encoder re-ranking, and temperature, and we evaluate six metrics: faithfulness, answer correctness, answer relevancy, context precision, context recall, and answer similarity. Chroma processes queries 13% faster, whereas Faiss yields higher retrieval precision, revealing a clear speed-accuracy trade-off. Naive fixed-length chunking with small windows and minimal overlap outperforms semantic segmentation while remaining the quickest option. Re-ranking provides modest gains in retrieval quality yet increases runtime by roughly a factor of 5, so its usefulness depends on latency constraints. These results help practitioners balance computational cost and accuracy when tuning RAG systems for transparent, up-to-date responses. Finally, we re-evaluate the top configurations with a corrective RAG workflow and show that their advantages persist when the model can iteratively request additional evidence. We obtain a near-perfect context precision (99%), which demonstrates that RAG systems can achieve extremely high retrieval accuracy with the right combination of hyperparameters, with significant implications for applications where retrieval quality directly impacts downstream task performance, such as clinical decision support in healthcare.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。