通过解耦参数空间，提升大模型对外部知识的准确使用与鲁棒性。
Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuning
- 解耦内部参数子空间，分别优化知识遵循与抗干扰能力。
- 在多个数据集上显著降低幻觉率，提升事实准确性。
- 适合需要精准引用外部知识的对话、问答等场景。
检索增强生成（RAG）通过引入外部检索知识，有效缓解了大语言模型（LLM）在幻觉生成和知识过时方面的问题。然而，现有方法缺乏对内部与外部知识融合的有效控制机制。受人类认知过程启发，我们提出Parenting框架，通过解耦、识别并针对性优化与知识遵循性和鲁棒性相关的参数子空间。具体而言，Parenting采用一种结合前向与反向传播信号的关键参数挖掘方法，定位代表不同能力的子空间；随后，采用类型定制化调优策略，对不同子空间施加特定且合适的优化，以实现对知识遵循性与鲁棒性的平衡提升。在多种数据集和模型上的大量实验验证了该方法的有效性与泛化能力。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) offers an effective solution to the issues faced by Large Language Models (LLMs) in hallucination generation and knowledge obsolescence by incorporating externally retrieved knowledge. However, existing methods lack effective control mechanisms for integrating internal and external knowledge. Inspired by human cognitive processes, we propose Parenting, a novel framework that decouples, identifies, and purposefully optimizes parameter subspaces related to adherence and robustness. Specifically, Parenting utilizes a key parameter mining method that combines forward and backward propagation signals to localize subspaces representing different capabilities. Then, Parenting employs a type-tailored tuning strategy, applying specific and appropriate optimizations to different subspaces, aiming to achieve a balanced enhancement of both adherence and robustness. Extensive experiments on various datasets and models validate the effectiveness and generalizability of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。