动态检索增强生成框架,提升大模型知识问答准确率与推理效率。
Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding
- 通过策略优化与自适应注意力评分,动态决定检索时机与内容。
- 在基准数据集上显著降低幻觉率,提升事实准确性与领域推理能力。
- 轻量级设计兼容任意Transformer模型,适合长文本与高并发场景。
我们提出一个综合性框架,通过动态检索策略与强化学习微调,提升检索增强生成(RAG)系统在知识密集型任务中的表现,包括开放域问答与复杂推理。该框架融合两项互补技术:政策优化的检索增强生成(PORAG),用于优化检索信息的利用;自适应令牌层注意力评分(ATLAS),根据上下文需求动态决定检索时机与内容。二者协同提升检索内容的利用率与相关性,增强事实准确性与响应质量。作为无需额外训练的轻量级方案,兼容任意基于Transformer的LLM,显著提升RAG场景下的输出准确率。我们还提出CRITIC方法,通过令牌重要性选择性压缩键值缓存,缓解长上下文应用中的内存瓶颈。框架集成测试时扩展技术,动态平衡推理深度与计算资源,并采用优化解码策略加速推理。在基准数据集上的实验表明,该框架有效减少幻觉、强化领域特定推理,相较传统RAG系统实现显著的效率与可扩展性提升。
原文摘要 · Abstract (English)
We present a comprehensive framework for enhancing Retrieval-Augmented Generation (RAG) systems through dynamic retrieval strategies and reinforcement fine-tuning. This approach significantly improves large language models on knowledge-intensive tasks, including opendomain question answering and complex reasoning. Our framework integrates two complementary techniques: Policy-Optimized RetrievalAugmented Generation (PORAG), which optimizes the use of retrieved information, and Adaptive Token-Layer Attention Scoring (ATLAS), which dynamically determines retrieval timing and content based on contextual needs. Together, these techniques enhance both the utilization and relevance of retrieved content, improving factual accuracy and response quality. Designed as a lightweight solution compatible with any Transformer-based LLM without requiring additional training, our framework excels in knowledge-intensive tasks, boosting output accuracy in RAG settings. We further propose CRITIC, a novel method to selectively compress key-value caches by token importance, mitigating memory bottlenecks in long-context applications. The framework also incorporates test-time scaling techniques to dynamically balance reasoning depth and computational resources, alongside optimized decoding strategies for faster inference. Experiments on benchmark datasets show that our framework reduces hallucinations, strengthens domain-specific reasoning, and achieves significant efficiency and scalability gains over traditional RAG systems. This integrated approach advances the development of robust, efficient, and scalable RAG systems across diverse applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。