用轻量适配器自动修正大模型幻觉，提升事实准确性。
AutoRAG-LoRA: Hallucination-Triggered Knowledge Retuning via Lightweight Adapters
- 通过低秩适配器和对比KL损失，动态修正生成内容。
- 在多个数据集上将幻觉率降低40%以上，保持高效推理。
- 适合需要高可靠性生成的场景，如医疗、法律问答。
大型语言模型在自然语言任务中表现出卓越的流畅性，但仍易产生幻觉——即事实性错误，影响实际部署中的可信度。本文提出AutoRAG-LoRA，一种基于轻量级LoRA适配器与KL正则化训练的检索增强生成框架，以应对大模型幻觉问题。该流程整合自动化提示重写、混合检索及低秩适配器微调，使生成结果基于检索到的证据。一个基于分类器与自评估的幻觉检测模块为输出分配置信度分数，并可触发反馈修正循环。该循环通过对比KL损失和适配器微调强制事实对齐。实验表明，AutoRAG-LoRA显著降低了事实漂移，同时保持模型的效率与模块化特性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable fluency across a range of natural language tasks, yet remain vulnerable to hallucinations - factual inaccuracies that undermine trust in real world deployment. We present AutoRAG-LoRA, a modular framework for Retrieval-Augmented Generation (RAG) that tackles hallucination in large language models through lightweight LoRA-based adapters and KL-regularized training. Our pipeline integrates automated prompt rewriting, hybrid retrieval, and low-rank adapter tuning to ground responses in retrieved evidence. A hallucination detection module, using both classifier-based and self-evaluation techniques, assigns confidence scores to generated outputs, triggering an optional feedback correction loop. This loop enforces factual alignment via contrastive KL loss and adapter fine tuning. We demonstrate that AutoRAG-LoRA significantly reduces the factual drift while preserving the efficiency and modularity of the model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。