用隐私知识库持续训练大模型并结合语义检索，有效减少幻觉。
Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained LLMs with RAG
- 用隐私知识库持续预训练基础模型，再加语义检索增强
- 处理隐私问题时性能提升最高达原模型两倍
- 适合需要高准确率的隐私相关问答场景
本文提出新方法，通过持续预训练基础大模型并引入语义检索增强层，以降低幻觉。该方法利用特定于隐私的知识库对模型进行持续学习，并在推理阶段融合检索结果，实现更精准的回答。实验表明，在处理隐私相关查询时，该方法相比未经微调的大模型，关键指标最高提升一倍，显著减少错误信息输出，提升回答的可信度和事实一致性。
原文摘要 · Abstract (English)
This paper presents new methods that have the potential to improve privacy process efficiency with LLM and RAG. To reduce hallucination, we continually pre-train the base LLM model with a privacy-specific knowledge base and then augment it with a semantic RAG layer. Our evaluations demonstrate that this approach enhances the model performance (as much as doubled metrics compared to out-of-box LLM) in handling privacy-related queries, by grounding responses with factual information which reduces inaccuracies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。