提升大模型在噪声信息下的问答鲁棒性
Towards Building a Robust Knowledge Intensive Question Answering Model with Large Language Models
- 通过数据增强微调提升模型抗噪能力
- 对比学习保留外部信息区分能力
- 适用于需要可靠知识推理的场景
大语言模型（LLMs）的发展显著提升了问答的智能水平和表达流畅性，检索增强技术也使模型能更好利用外部信息。然而，检索结果中的噪声和错误对模型的鲁棒性构成挑战。为此，本文基于机器阅读理解数据集构建了一个新数据集，模拟关键信息缺失、噪声和冲突等多种干扰场景。为应对噪声导致的准确率下降，提出一种基于数据增强的微调方法以增强模型鲁棒性；同时引入对比学习，保持模型对外部信息的区分能力。在现有LLMs及所提方法上进行实验，由GPT-4评估结果表明，该方法在提升鲁棒性的同时强化了模型的信息辨别能力。
原文摘要 · Abstract (English)
The development of LLMs has greatly enhanced the intelligence and fluency of question answering, while the emergence of retrieval enhancement has enabled models to better utilize external information. However, the presence of noise and errors in retrieved information poses challenges to the robustness of LLMs. In this work, to evaluate the model's performance under multiple interferences, we first construct a dataset based on machine reading comprehension datasets simulating various scenarios, including critical information absence, noise, and conflicts. To address the issue of model accuracy decline caused by noisy external information, we propose a data augmentation-based fine-tuning method to enhance LLM's robustness against noise. Additionally, contrastive learning approach is utilized to preserve the model's discrimination capability of external information. We have conducted experiments on both existing LLMs and our approach, the results are evaluated by GPT-4, which indicates that our proposed methods improve model robustness while strengthening the model's discrimination capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。