让大模型在问答时自动适应专业领域，提升知识检索效果
Predict the Retrieval! Test time adaptation for Retrieval Augmented Generation
- 推理时动态调整模型参数，根据检索内容自适应优化
- 在6个专业领域上显著优于基线RAG系统
- 适合需要精准领域知识的问答场景
检索增强生成（RAG）通过整合外部知识，显著提升了大语言模型的问答能力。然而，在适配专业领域时，由于分布偏移问题，模型泛化性能下降。本文提出TTARAG，一种测试时自适应方法，在推理过程中动态更新语言模型参数，以提升专业领域下的RAG表现。该方法通过让模型学习预测检索内容，实现对目标领域的自动参数调整。在六个专业领域上的大量实验表明，TTARAG相比基线RAG系统有显著性能提升。代码已开源：https://github.com/sunxin000/TTARAG。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has emerged as a powerful approach for enhancing large language models' question-answering capabilities through the integration of external knowledge. However, when adapting RAG systems to specialized domains, challenges arise from distribution shifts, resulting in suboptimal generalization performance. In this work, we propose TTARAG, a test-time adaptation method that dynamically updates the language model's parameters during inference to improve RAG system performance in specialized domains. Our method introduces a simple yet effective approach where the model learns to predict retrieved content, enabling automatic parameter adjustment to the target domain. Through extensive experiments across six specialized domains, we demonstrate that TTARAG achieves substantial performance improvements over baseline RAG systems. Code available at https://github.com/sunxin000/TTARAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。