arXiv:2507.16754cs.SEcs.AI2025-07

用自适应检索提升大模型代码回答可靠性,覆盖新旧问题

Never Come Up Empty: Adaptive HyDE Retrieval for Improving LLM Developer Support

  • 设计7种RAG方案并优化HyDE结合完整答案上下文
  • 通过动态降低相似度阈值,提升对新问题的召回率
  • 在4个开源模型上验证,显著提升回答准确性和细节

大型语言模型在辅助开发者解决代码问题方面展现出潜力,但存在生成不可靠答案的风险。为缓解此问题,检索增强生成(RAG)被提出以减少大模型的幻觉。本文构建了一个包含超过300万条与Java和Python相关的已接受答案的Stack Overflow帖子语料库,探索多种RAG管道设计来回答开发者问题,并评估其生成准确可靠响应的效果。具体而言,(1) 设计并评估了7种不同的RAG管道及其63种变体,用于回答历史上有相似匹配的问题;(2) 针对无相近历史匹配的新问题,通过在检索时自动降低相似度阈值,提高找到部分相关上下文的概率,从而增强对未见情况的覆盖能力。研究发现,将假设文档嵌入(HyDE)与完整答案上下文结合的RAG管道在检索和回答相似内容方面表现最佳。最后,将最优RAG管道应用于4个开源大模型,并与零样本性能对比。结果表明,该最优RAG管道在所有模型上均一致优于零样本基线,在大模型评分器下获得更高的帮助性、正确性和细节得分,证明其能有效提升各类开发者查询的回答质量,涵盖已见与新问题。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have shown promise in assisting developers with code-related questions; however, LLMs carry the risk of generating unreliable answers. To address this, Retrieval-Augmented Generation (RAG) has been proposed to reduce the unreliability (i.e., hallucinations) of LLMs. However, designing effective pipelines remains challenging due to numerous design choices. In this paper, we construct a retrieval corpus of over 3 million Java and Python related Stack Overflow posts with accepted answers, and explore various RAG pipeline designs to answer developer questions, evaluating their effectiveness in generating accurate and reliable responses. More specifically, we (1) design and evaluate 7 different RAG pipelines and 63 pipeline variants to answer questions that have historically similar matches, and (2) address new questions without any close prior matches by automatically lowering the similarity threshold during retrieval, thereby increasing the chance of finding partially relevant context and improving coverage for unseen cases. We find that implementing a RAG pipeline combining hypothetical-documentation-embedding (HyDE) with the full-answer context performs best in retrieving and answering similarcontent for Stack Overflow questions. Finally, we apply our optimal RAG pipeline to 4 open-source LLMs and compare the results to their zero-shot performance. Our findings show that RAG with our optimal RAG pipeline consistently outperforms zero-shot baselines across models, achieving higher scores for helpfulness, correctness, and detail with LLM-as-a-judge. These findings demonstrate that our optimal RAG pipelines robustly enhance answer quality for a wide range of developer queries including both previously seen and novel questions across different LLMs

大模型代码生成RAG检索增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。