用机器学习预测RAG检索质量,提升生成内容准确性。
Quality analysis and evaluation prediction of RAG retrieval based on machine learning algorithms
- 基于特征工程与粒子群优化的XGBoost回归模型
- 文档相关性与答案质量正相关(0.66),冗余度与多样性负相关(-0.89)
- 模型在多个指标上优于传统方法,适合优化RAG系统
随着大语言模型的快速发展,基于外部知识增强生成的技术因其提升输出准确性的能力而广泛应用。然而,系统性能高度依赖于检索模块的质量。若检索结果与用户需求相关性低或含噪声信息,将直接导致生成内容失真。针对现有模型处理表格特征的性能瓶颈,本文提出一种基于特征工程与粒子群优化的XGBoost机器学习回归模型。相关性分析显示,answer_quality与doc_delevance呈正相关(0.66),表明文档相关性对答案质量有显著正向影响;语义相似性、冗余度与多样性之间存在强负相关(分别为-0.89和-0.88),说明三者间存在权衡关系:前两者上升时,多样性显著下降。实验对比决策树、AdaBoost等模型的结果表明,VMD PSO BiLSTM模型在所有评估指标上均表现更优,其MSE、RMSE、MAE、MAPE显著更低,R2值更高,体现出更强的预测精度、稳定性和数据解释能力。该成果为优化RAG检索质量、提升生成效果提供了有效路径,在推动相关技术落地应用方面具有重要意义。
原文摘要 · Abstract (English)
With the rapid evolution of large language models, retrieval enhanced generation technology has been widely used due to its ability to integrate external knowledge to improve output accuracy. However, the performance of the system is highly dependent on the quality of the retrieval module. If the retrieval results have low relevance to user needs or contain noisy information, it will directly lead to distortion of the generated content. In response to the performance bottleneck of existing models in processing tabular features, this paper proposes an XGBoost machine learning regression model based on feature engineering and particle swarm optimization. Correlation analysis shows that answer_quality is positively correlated with doc_delevance by 0.66, indicating that document relevance has a significant positive effect on answer quality, and improving document relevance may enhance answer quality; The strong negative correlations between semantic similarity, redundancy, and diversity were -0.89 and -0.88, respectively, indicating a tradeoff between semantic similarity, redundancy, and diversity. In other words, as the former two increased, diversity significantly decreased. The experimental results comparing decision trees, AdaBoost, etc. show that the VMD PSO BiLSTM model is superior in all evaluation indicators, with significantly lower MSE, RMSE, MAE, and MAPE compared to the comparison model. The R2 value is higher, indicating that its prediction accuracy, stability, and data interpretation ability are more outstanding. This achievement provides an effective path for optimizing the retrieval quality and improving the generation effect of RAG system, and has important value in promoting the implementation and application of related technologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。