让小模型在问答中更准更稳，通过优化选择更好答案
RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization
- 用多轮提示+拒绝采样生成高质量推理过程
- 通过对比偏好筛选，提升优选答案的置信度差距
- 适合资源受限场景下需要可靠回答的小模型
大语言模型虽表现优异，但计算成本高、延迟大，限制了在资源受限环境中的部署。相比之下，小规模语言模型（SLMs）效率更高，却难以捕捉不断变化的真实世界知识。检索增强生成（RAG）通过引入外部知识来缓解问题，但不完善的检索会引入干扰噪声，误导SLMs。本文提出RoseRAG，一种面向SLMs的鲁棒RAG框架，基于边界感知偏好优化。RoseRAG采用多轮提示进行详细推理，使用拒绝采样获取高质量解释，并通过对比偏好选择机制，最大化优选与非优选输出之间的概率差距。将这些组件整合进边界感知优化流程后，显著提升了SLMs在RAG应用中的准确性和可靠性。在三个开放域问答基准上的实验表明，RoseRAG显著超越现有最先进方法。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings. In contrast, small-scale LLMs (SLMs) are more efficient yet struggle to capture evolving real-world knowledge. Retrieval-augmented generation (RAG) helps by integrating external knowledge, but imperfect retrieval can introduce distracting noise that misleads SLMs. We propose RoseRAG, a robust RAG framework for SLMs via Margin-aware Preference Optimization. RoseRAG employs multi-turn prompting for detailed reasoning, rejection sampling for high-quality explanations, and contrastive preference selection to refine responses by maximizing the likelihood gap between preferred and non-preferred outputs. By integrating these components into a margin-aware optimization process, RoseRAG robustly enhances the accuracy and reliability of SLMs for RAG applications. Extensive experiments on three open-domain question answering benchmarks indicate that our innovative RoseRAG surpasses state-of-the-art baselines significantly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。